Text Topic Modeling Research Articles

Recognition of Urban Functions and Mixed Use Based on Residents’ Movement and Topic Generation Model: The Case of Wuhan, China

The rapid evolution of cities has brought new challenges to urban planning and management. The accurate evaluation of urban functional structure and mixed use is critical, especially at a fine scale such as by blocks. The composition and mixing of urban spatial functions calculated by remote sensing and statistics are non-quantitative and undetailed. The text topic models are often applied to process text data, but are rarely used to mine semantic information in quantitative data. Therefore, this paper attempts to carry out research on the recognition of urban functions and mixed use using a text topic generation model based on resident mobile data. First, the area within Wuhan Third Ring Road was divided into 2451 units at a grid size of 500 m × 500 m. The histogram-latent Dirichlet allocation (H-LDA) and information entropy were applied to assign different grid units to correct the functional topics and topic information entropy (TIE). Second, the functional categories of different analysis units were calculated using the point of interest (POI), frequency density (FD) and category proportion (CP) indicators, while the functional information entropy (FIE) based on the POI was calculated. Then, the urban functions and mixtures identified by the two kinds of data were compared and analyzed. Finally, referring to the geographic information in streetscape map and applying correlation analysis, the function and mixing results obtained from the experiment were verified. Studies have shown that the H-LDA model can identify bridges, which the POI data have shown is challenging to identify without attributes such as length. The function recognition accuracy of the H-LDA model is 89.3%, which is higher than K-means algorithm and Word2vec models. The correlation coefficient between FIE and TIE is 0.587, indicating that both are highly correlated. These explain the accuracy and rationality of identifying city functions and mixtures based on the H-LDA model. The H-LDA model can be applied to functional computing and fine-scale urban mixed function planning.

A Pseudo-document-based Topical N-grams model for short texts

In recent years, short text topic modeling has drawn considerable attentions from interdisciplinary researchers. Various customized topic models have been proposed to tackle the semantic sparseness nature of short texts. Most (if not all) of them follow the bag-of-words assumption, which, however, is not adequate since word order and phrases are often critical to capturing the meaning of texts. On the other hand, while some existing topic models are sensitive to word order, they do not perform well on short texts due to the severe data sparseness. To address these issues, we propose the Pseudo-document-based Topical N-Grams model (PTNG), which alleviates the data sparsity problem of short texts while is sensitive to word order. Extensive experiments on three real-world data sets with state-of-the-art baselines demonstrate the high quality of topics learned by PTNG according to UCI coherence scores and more discriminative semantic representation of short texts according to classification results.

Short Text Topic Modeling Techniques, Applications, and Performance: A Survey

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm.

Heterogeneous-Length Text Topic Modeling for Reader-Aware Multi-Document Summarization

More and more user comments like Tweets are available, which often contain user concerns. In order to meet the demands of users, a good summary generating from multiple documents should consider reader interests as reflected in reader comments. In this article, we focus on how to generate a summary from multi-document documents by considering reader comments, named as reader-aware multi-document summarization (RA-MDS). We present an innovative topic-based method for RA-MDA, which exploits latent topics to obtain the most salient and lessen redundancy summary from multiple documents. Since finding latent topics for RA-MDS is a crucial step, we also present a Heterogeneous-length Text Topic Modeling (HTTM) to extract topics from the corpus that includes both news reports and user comments, denoted as heterogeneous-length texts. In this case, the latent topics extract by HTTM cover not only important aspects of the event, but also aspects that attract reader interests. Comparisons on summary benchmark datasets also confirm that the proposed RA-MDS method is effective in improving the quality of extracted summaries. In addition, experimental results demonstrate that the proposed topic modeling method outperforms existing topic modeling algorithms.

Dirichlet Multinomial Mixture with Variational Manifold Regularization: Topic Modeling over Short Texts

Conventional topic models suffer from a severe sparsity problem when facing extremely short texts such as social media posts. The family of Dirichlet multinomial mixture (DMM) can handle the sparsity problem, however, they are still very sensitive to ordinary and noisy words, resulting in inaccurate topic representations at the document level. In this paper, we alleviate this problem by preserving local neighborhood structure of short texts, enabling to spread topical signals among neighboring documents, so as to correct the inaccurate topic representations. This is achieved by using variational manifold regularization, constraining the close short texts should have similar variational topic representations. Upon this idea, we propose a novel Laplacian DMM (LapDMM) topic model. During the document graph construction, we further use the word mover’s distance with word embeddings to measure document similarities at the semantic level. To evaluate LapDMM, we compare it against the state-of-theart short text topic models on several traditional tasks. Experimental results demonstrate that our LapDMM achieves very significant performance gains over baseline models, e.g., achieving even about 0.2 higher scores on clustering and classification tasks in many cases.

A case study in text mining: Textual analysis of the Territorial Papers

Abstract The Territorial Papers of the United States are a valuable and underused resource containing almost 10,000 documents written between 1789 and 1848 about the formation of new sovereign states from US territory. These communications between the federal government and frontier settlers comprise the actual discourse of the nation’s expansion over six decades. Digitizing the Territorial Papers permits the possibility of analyzing the entire corpus globally. Text mining and topic modeling methods give us a lens on the language patterns through which new state governments and the expanding nation were formed. An initial statistical analysis of the textual information provides a visualization of content, helps discern how ideals about governance emerged, and lays the foundation for developing more sophisticated hypotheses and theoretical constructs.

Incorporating word embeddings into topic modeling of short text

Short texts have become the prevalent format of information on the Internet. Inferring the topics of this type of messages becomes a critical and challenging task for many applications. Due to the length of short texts, conventional topic models (e.g., latent Dirichlet allocation and its variants) suffer from the severe data sparsity problem which makes topic modeling of short texts difficult and unreliable. Recently, word embeddings have been proved effective to capture semantic and syntactic information about words, which can be used to induce similarity measures and semantic correlations among words. Enlightened by this, in this paper, we design a novel model for short text topic modeling, referred as Conditional Random Field regularized Topic Model (CRFTM). CRFTM not only develops a generalized solution to alleviate the sparsity problem by aggregating short texts into pseudo-documents, but also leverages a Conditional Random Field regularized model that encourages semantically related words to share the same topic assignment. Experimental results on two real-world datasets show that our method can extract more coherent topics, and significantly outperform state-of-the-art baselines on several evaluation metrics.

Experimental explorations on short text topic mining between LDA and NMF based Schemes

Learning topics from short texts has become a critical and fundamental task for understanding the widely-spread streaming social messages, e.g., tweets, snippets and questions/answers. Up to date, there are two distinctive topic learning schemes: generative probabilistic graphical models and geometrically linear algebra approaches, with LDA and NMF being the representative works, respectively. Since these two methods both could uncover the latent topics hidden in the unstructured short texts, some interesting doubts are coming to our minds that which one is better and why? Are there any other more effective extensions? In order to explore valuable insights between LDA and NMF based learning schemes, we comprehensively conduct a series of experiments into two parts. Specifically, the basic LDA and NMF are compared with different experimental settings on several public short text datasets in the first part which would exhibit that NMF tends to perform better than LDA; in the second part, we propose a novel model called “Knowledge-guided Non-negative Matrix Factorization for Better Short Text Topic Mining” (abbreviated as KGNMF), which leverages external knowledge as a semantic regulator with low-rank formalizations, yielding up a time-efficient algorithm. Extensive experiments are conducted on three representative corpora with currently typical short text topic models to demonstrate the effectiveness of our proposed KGNMF. Overall, learning with NMF-based schemes is another effective manner in short text topic mining in addition to the popular LDA-based paradigms.

Topic modeling for analyzing open-ended survey responses

ABSTRACTOpen-ended responses are widely used in market research studies. Processing of such responses requires labour-intensive human coding. This paper focuses on unsupervised topic models and tests their ability to automate the analysis of open-ended responses. Since state-of-the-art topic models struggle with the shortness of open-ended responses, the paper considers three novel short text topic models: Latent Feature Latent Dirichlet Allocation, Biterm Topic Model and Word Network Topic Model. The models are fitted and evaluated on a set of real-world open-ended responses provided by a market research company. Multiple components such as topic coherence and document classification are quantitatively and qualitatively evaluated to appraise whether topic models can replace human coding. The results suggest that topic models are a viable alternative for open-ended response coding. However, their usefulness is limited when a correct one-to-one mapping of responses and topics or the exact topic distribution is needed.

Hedonic and eudaimonic well-being: A psycholinguistic view

Well-being as an intangible, philosophical, and multi-faceted phenomenon is hard to measure. By taking a psycholinguistic expression of well-being, we measure how tourists' experiencing holiday destinations affects their well-being states. The well-being state includes increases or decreases in Hedonia and Eudaimonia as a result of destination experiences. We apply text topic modelling to analyse big Web 2.0 datasets. These include tourists’ self-reports of their experiences during their visit to destination countries collected from Travelblog.org weblog. The findings from a Global sample, New Zealand, and France are compared to generalise Hedonia and Eudaimonia conceptualisations across these destinations. The outputs also characterise experiences with maximum well-being and ill-being that include both Hedonia and Eudaimonia. Managerial implications include how a destination image may be managed according to desired well-being states. This also helps tourists make informed decisions about their holiday destinations.

Filtering out the noise in short text topic modeling

Nowadays, massive short texts, such as social media posts and newspaper titles, are available on the Internet. Analyzing these short texts is very significant for many content analysis tasks. However, the commonly used text analysis tools, i.e., topic models, lose effectiveness on short texts because of the sparsity and noise problems. Recent topic models mainly attempt to solve the sparsity problem, but neglect the noise issue. To address this, we propose a common semantics topic model (CSTM) in this paper. The key idea is to introduce a new type of topic, namely common topic, to gather the noise words. The experimental results on real-world datasets indicate that our CSTM outperforms the existing short text topic models on the traditional tasks.

The Architecture for Optimizing Semantic Topical Coherence in Gujarati Text Topic Modeling

The Architecture for Optimizing Semantic Topical Coherence in Gujarati Text Topic Modeling

GLTM: A Global and Local Word Embedding-Based Topic Model for Short Texts

Short texts have become a kind of prevalent source of information, and discovering topical information from short text collections is valuable for many applications. Due to the length limitation, conventional topic models based on document-level word co-occurrence information often fail to distill semantically coherent topics from short text collections. On the other hand, word embeddings as a powerful tool have been successfully applied in natural language processing. Word embeddings trained on large corpus are encoded with general semantic and syntactic information of words, and hence they can be leveraged to guide topic modeling for short text collections as supplementary information for sparse co-occurrence patterns. However, word embeddings are trained on large external corpus and the encoded information is not necessarily suitable for training data set of topic models, which is ignored by most existing models. In this article, we propose a novel global and local word embedding-based topic model (GLTM) for short texts. In the GLTM, we train global word embeddings from large external corpus and employ the continuous skip-gram model with negative sampling (SGNS) to obtain local word embeddings. Utilizing both the global and local word embeddings, the GLTM can distill semantic relatedness information between words which can be further leveraged by Gibbs sampler in the inference process to strengthen semantic coherence of topics. Compared with five state-of-the-art short text topic models on four real-world short text collections, the proposed GLTM exhibits the superiority in most cases.

Short text topic modeling by exploring original documents

Topic modeling for short texts faces a tough challenge, owing to the sparsity problem. An effective solution is to aggregate short texts into long pseudo-documents before training a standard topic ...

BTM: Topic Modeling over Short Texts

Short texts are popular on today's web, especially with the emergence of social media. Inferring topics from large scale short texts becomes a critical but challenging task for many content analysis tasks. Conventional topic models such as latent Dirichlet allocation (LDA) and probabilistic latent semantic analysis (PLSA) learn topics from document-level word co-occurrences by modeling each document as a mixture of topics, whose inference suffers from the sparsity of word co-occurrence patterns in short texts. In this paper, we propose a novel way for short text topic modeling, referred as biterm topic model (BTM). BTM learns topics by directly modeling the generation of word co-occurrence patterns (i.e., biterms) in the corpus, making the inference effective with the rich corpus-level information. To cope with large scale short text data, we further introduce two online algorithms for BTM for efficient topic learning. Experiments on real-word short text collections show that BTM can discover more prominent and coherent topics, and significantly outperform the state-of-the-art baselines. We also demonstrate the appealing performance of the two online BTM algorithms on both time efficiency and topic learning.

The divided communities of shared concerns: Mapping the intellectual structure of e-Health research in social science journals

PurposeSocial scientific approach has become an important approach in e-Health studies over the past decade. However, there has been little systematical examination of what aspects of e-Health social scientists have studied and how relevant and informative knowledge has been produced and diffused by this line of inquiry. This study performed a systematic review of the body of e-Health literature in mainstream social science journals over the past decade by testing the applicability of a 5A categorization (i.e., access, availability, appropriateness, acceptability, and applicability), proposed by the U.S. Department of Health and Human Services, as a framework for understanding social scientific research in e-Health. MethodsThis study used a quantitative, bottom-up approach to review the e-Health literature in social sciences published from 2000 to 2009. A total of 3005 e-Health studies identified from two social sciences databases (i.e., Social Sciences Citation Index and Arts & Humanities Citation Index) were analyzed with text topic modeling and structural analysis of co-word network, co-citation network, and scientific food web. ResultsThere have been dramatic increases in the scale of e-Health studies in social sciences over the past decade in terms of the numbers of publications, journal outlets and participating disciplines. The results empirically confirm the presence of the 5A clusters in e-Health research, with the cluster of applicability as the dominant research area and the cluster of availability as the major knowledge producer for other clusters. The network analysis also reveals that the five distinctive clusters share much more in common in research concerns than what e-Health scholars appear to recognize. ConclusionsIt is time to explicate and, more importantly, tap into the shared concerns cutting across the seemingly divided scholarly communities. In particular, more synergy exercises are needed to promote adherence of the field.

Topic Modeling for Wikipedia Link Disambiguation

Many articles in the online encyclopedia Wikipedia have hyperlinks to ambiguous article titles; these ambiguous links should be replaced with links to unambiguous articles, a process known as disambiguation. We propose a novel statistical topic model based on link text, which we refer to as the Link Text Topic Model (LTTM), that we use to suggest new link targets for ambiguous links. To evaluate our model, we describe a method for extracting ground truth for this link disambiguation task from edits made to Wikipedia in a specific time period. We use this ground truth to demonstrate the superiority of LTTM over other existing link- and content-based approaches to disambiguating links in Wikipedia. Finally, we build a web service that uses LTTM to make suggestions to human editors wanting to fix ambiguous links in Wikipedia.