Probabilistic Topic Model Research Articles

Integrating Topic, Sentiment, and Syntax for Modeling Online Reviews: A Topic Model Approach

Analyzing product online reviews has drawn much interest in the academic field. In this research, a new probabilistic topic model, called tag sentiment aspect models (TSA), is proposed on the basis of Latent Dirichlet allocation (LDA), which aims to reveal latent aspects and corresponding sentiment in a review simultaneously. Unlike other topic models which consider words in online reviews only, syntax tags are taken as visual information and, in this research, as a kind of widely used syntax information, part-of-speech (POS) tags are first reckoned. Specifically, POS tags are integrated into three versions of implementation in consideration of the fact that words with different POS tags might be utilized to express consumers' opinions. Also, the proposed TSA is one unsupervised approach and only a small number of positive and negative words are required to confine different priors for training. Finally, two big datasets regarding digital SLR and laptop are utilized to evaluate the performance of the proposed model in terms of sentiment classification and aspect extraction. Comparative experiments show that the new model can not only achieve promising results on sentiment classification but also leverage the performance on aspect extraction.

Distributed Gaussian mixture model for monitoring plant-wide processes with multiple operating modes

For large-scale plant-wide processes with multiple operating conditions, a distributed Gaussian mixture modeling and monitoring mechanism is proposed. To overcome the deficient prior modeling knowledge for complex process, a two-dimensional probabilistic topic model based technique named Bayesian co-clustering method is developed to simultaneously conduct the sub-block division and operating mode recognition. With the obtained sub-blocks and operating modes, a global Gaussian mixture model is first built and then several local Gaussian mixture models are extracted and applied for distributed monitoring of plant-wide processes. By conducting distributed modeling and monitoring, both global and local process changes can be reflected and the fault region can be localized more easily for further analyses. The feasibility and effectiveness of the proposed method is confirmed through a numerical example and the Tennessee Eastman benchmark process.

Efficient algorithms for graph regularized PLSA for probabilistic topic modeling

Probabilistic latent semantic analysis (PLSA) is a popular data analysis method with the objective to discover the underlying semantic structure of input data. In this work, we describe a method for probabilistic topic analysis in image and text based on a new representation of graph-regularized PLSA (GPLSA). In GPLSA, data entities are mapped to an undirected graph, where similarities between topic compositions on the graph are measured by the divergence between discrete probabilities. Such divergence is essentially incorporated as a graph-regularizer that augments the original PLSA algorithm. Furthermore, we extend the GPLSA algorithms to multiple data modalities based on the connections between data entities of each modality. We propose efficient multiplicative iterative algorithms for GPLSA with three popular regularizers, namely ℓ1, ℓ2 and symmetric KL divergences. In each case, we derive simple efficient numerical solutions that require only matrix arithmetic operations during the optimization. Experimental results demonstrate the efficacy of GPLSA over state-of-the-art methods.

Multimodal Probabilistic Latent Semantic Analysis for Sentinel-1 and Sentinel-2 Image Fusion

Probabilistic topic models have recently shown a great potential in the remote sensing image fusion field, which is particularly helpful in land-cover categorization tasks. This letter first studies the application of probabilistic latent semantic analysis (pLSA) and latent Dirichlet allocation to remote sensing synthetic aperture radar (SAR) and multispectral imaging (MSI) unsupervised land-cover categorization. Then, a novel pLSA-based image fusion approach is presented, which pursues to uncover multimodal feature patterns from SAR and MSI data in order to effectively fuse and categorize Sentinel-1 and Sentinel-2 remotely sensed data. Experiments conducted over two different data sets reveal the advantages of the proposed approach for unsupervised land-cover categorization tasks.

Learning Unsupervised Representations from Biomedical Text

IntroductionHealthcare settings are becoming increasingly technological. Interactions/events involving healthcare providers and the patients they service are captured as digital text. Healthcare organizations are amassing increasingly large/complex collections of biomedical text data. Researchers and policy makers are beginning to explore these text data holdings for structure, patterns, and meaning.
 Objectives and ApproachEMRALD is a primary care electronic medical record (EMR) database, comprised of over 40 family medicine clinics, nearly 400 primary care physicians and over 500,000 patients. EMRALD includes full-chart extractions, including all clinical narrative information/data in a variety of fields.
 The input data (raw text strings) are discrete, sparse and high dimensional. We assessed scalable statistical models for high dimensional discrete data, including fitting, assessing and exploring models from three broad statistical areas: i) matrix factorization/decomposition models ii) probabilistic topic models and iii) word-vector embedding models.
 ResultsEMRALD is comprised of 12 text data streams. EMRALD text data is structured into 84 million clinical notes (3.5 billion word/language tokens) and is approximately 18Gb in storage size. We employ a “text as data” pipeline, i) mapping raw strings to sequences of word/language tokens, ii) mapping token sequences to numeric arrays, and finally iii) using numeric arrays as inputs to statistical models.
 Fitted topic models yield useful thematic summaries of the EMRALD corpora. Topics discovered reflect core responsibilities of primary care physicians (e.g. women’s health, pain management, nutrition/diet, etc.).
 Fitted vector embedding models capture structure of discourse/syntax. Related words are mapped to similar locations of vector spaces. Analogical reasoning is possible in the embedding space.
 Conclusion/Implications“Text as data” requires an understanding of statistical models for discrete, sparse, high dimensional data. We fit a variety of unsupervised statistical models to biomedical text data. Preliminary results suggest that the learned low dimensional representations of the biomedical text data are effective at uncovering meaningful patterns/structure.

A comprehensive approach to reviewing latent topics addressed by literature across multiple disciplines

This paper proposes an approach to capturing and reviewing scientific literature addressing latent topics across multiple scientific fields. As latent topics like moral values are affected by word polysemy and synonymy, a traditional keyword-based approach is often ineffective and therefore inappropriate. As a result, scientific literature addressing latent topics tends to be fragmented thereby constraining efforts to address similar and complementary research challenges. A novel approach to reviewing the literature by utilizing both semantic fields and probabilistic topic models has therefore been developed. We illustrate this approach by reviewing the literature addressing the value justice in the energy sector and compare this with a regular keyword-based approach. The new approach results in a more complete overview of the relevance of energy justice as compared to the traditional keyword-based approach. This novel approach can be applied to other latent topics including other values or phenomena such as societal resistance to technologies, thereby leading to an increased understanding of existing relevant literature and the identification of new areas of research.

A Topical Exploration of the Intellectual Development of Decision Sciences 1975–2016

ABSTRACTThis article reviews Decision Sciences journal articles and metadata to analyze its intellectual tradition. Text analytics is used with probabilistic topic modeling. The topical structure of the journal is reviewed by topic definition and popularity, with correlations. Fifty research topics or themes involving a wide range of quantitative methods and decision‐making practice were selected. Functional areas were also examined. The evolution of topics since 1975 is noted. There is clear indication that journal coverage has evolved. In early years, emphasis was on quantitative modeling methods and relevant methodologies. More recently new research areas to include supply chain management, marketing, service management, and health care are more noted. Some topics are highly correlated. We find that this evolution reflects the changes occurring in business and decision‐making environment. The article discusses external and internal factors important in shaping the journal's topical trajectory. Unique challenges in analyzing text data are discussed. Latent Dirichlet Allocation, an unsupervised Bayesian approach for statistical topic modeling, is applied to 1,698 research articles from Decision Sciences over the period from 1975 to 2016. This approach is found to be useful to discover journal topical trends. Potential for other applications in decision sciences is discussed.

Stream-based live public opinion monitoring approach with adaptive probabilistic topic model

Public opinion monitoring, also known as first story detection, is defined within the topic detection and tracking on a particular Internet news event. Generally, it is used to find news propagation. Traditional method adopts text matching to address opinion monitoring. But it has some limitations such as hidden and latent topic discovery and incorrect relevance ranking of matching results on large-scale data. In this paper, we propose three solutions to live public opinion monitoring: simple keyword computing and matching, simple probabilistic topic computing and matching, and stream-based live probabilistic topic computing and matching. We point out the disadvantages of the first two solutions such as semantic matching and low efficiency on timely big data. Stream-based real-time topic computing and topic matching with query-time document and field boosting are proposed to make substantial improvements. Finally, our topic computing and matching experiments with crawled historical Netease news records show that our approaches are effective and efficient.

Examining the Impact of Contextual Ambiguity on Search Advertising Keyword Performance: A Topic Model Approach

In this paper, we explore how the contextual ambiguity of a search can affect a keyword's performance. The context of consumer search is often unobserved and the prediction of it can be nontrivial. Consumers arrive at search engines with diverse interests, and their search context may vary even when they are searching using the same keyword. In our study, we propose an automatic way of examining keyword contextual ambiguity based on probabilistic topic models from machine learning and computational linguistics. We examine the effect of contextual ambiguity on keyword performance using a hierarchical Bayesian approach that allows for topic-specific effects and nonlinear position effects, and jointly models click-through rate (CTR) and ad position (rank). We validate our study using a novel data set from a major search engine that contains information on consumer click activities for 2,625 distinct keywords across multiple product categories from 10,000 impressions. We find that consumer click behavior varies significantly across keywords, and such variation can be partially explained by keyword category and the contextual ambiguity of keywords. Specifically, higher contextual ambiguity is associated with higher CTR on top-positioned ads, but also a faster decay in CTR with screen position. Therefore, the overall effect of contextual ambiguity on CTR varies across positions. Our study has the potential to help advertisers design keyword portfolios and bidding strategy by extracting contextual ambiguity and other semantic characteristics of keywords based on large-scale analytics from unstructured data. It can also help search engines improve the quality of displayed ads in response to a consumer search query.

The economic effects of U.S. presidential tax communication: Evidence from a correlated topic model

We combine a probabilistic topic model and a dictionary-based sentiment analysis to construct a time series, which indicates when and how (positive vs. negative) the U.S. president communicates his tax policy news to the public. The econometric analyses show that optimistic tax policy statements stimulate consumption, investment, and output, even after controlling for tax foresight. We also find that consumer sentiment reacts positively to more optimistic tax news, suggesting that sentiment plays an important role in the transmission from U.S. presidential tax policy communication to economic activity.

Scene Classification Based on the Sparse Homogeneous–Heterogeneous Topic Feature Model

High spatial resolution (HSR) imagery scene classification has been the subject of increased interest in recent years, and has great potential for many applications, such as urban functional analysis. Rooted in natural information processing, the use of the probabilistic topic model (PTM) to capture latent topics to represent HSR images has been an effective way to bridge the semantic gap. However, how to effectively discover discriminative information to recognize the HSR scenes is a challenging task. In this paper, the sparse homogeneous–heterogeneous topic feature model (SHHTFM) is proposed for HSR image scene classification. Differing from the conventional PTM-based scene classification methods, which utilize only heterogeneous features, SHHTFM explores the effect of the homogeneous information. Based on the union of uniform grid sampling and simple linear iterative clustering superpixel sampling, SHHTFM exploits both the heterogeneous and homogeneous information. After separately mining different types of low-level features and latent topics, the sparse topic inference procedure of SHHTFM further improves the fusion of the sparse heterogeneous and homogeneous topics. In addition, multisource geographical data are effectively integrated, where the water and vegetation boundaries define a more accurate way to restrict the boundaries of different scenes, and are then combined with the road network data to further improve the scene annotation performance. This provides more reliable and applicable results for us to better understand the complex scenes. The experimental results obtained with two HSR image classification data sets and an HSR image annotation data set demonstrate that the proposed SHHTFM framework can solve the scene classification problem, with a high classification accuracy as well as a high time efficiency.

Towards Training Probabilistic Topic Models on Neuromorphic Multi-Chip Systems

Probabilistic topic models are popular unsupervised learning methods, including probabilistic latent semantic indexing (pLSI) and latent Dirichlet allocation (LDA). By now, their training is implemented on general purpose computers (GPCs), which are flexible in programming but energy-consuming. Towards low-energy implementations, this paper investigates their training on an emerging hardware technology called the neuromorphic multi-chip systems (NMSs). NMSs are very effective for a family of algorithms called spiking neural networks (SNNs). We present three SNNs to train topic models.The first SNN is a batch algorithm combining the conventional collapsed Gibbs sampling (CGS) algorithm and an inference SNN to train LDA. The other two SNNs are online algorithms targeting at both energy- and storage-limited environments. The two online algorithms are equivalent with training LDA by using maximum-a-posterior estimation and maximizing the semi-collapsed likelihood, respectively.They use novel, tailored ordinary differential equations for stochastic optimization. We simulate the new algorithms and show that they are comparable with the GPC algorithms, while being suitable for NMS implementation. We also propose an extension to train pLSI and a method to prune the network to obey the limited fan-in of some NMSs.

Diagnosing and Improving Topic Models by Analyzing Posterior Variability

Bayesian inference methods for probabilistic topic models can quantify uncertainty in the parameters, which has primarily been used to increase the robustness of parameter estimates. In this work, we explore other rich information that can be obtained by analyzing the posterior distributions in topic models. Experimenting with latent Dirichlet allocation on two datasets, we propose ideas incorporating information about the posterior distributions at the topic level and at the word level. At the topic level, we propose a metric called topic stability that measures the variability of the topic parameters under the posterior. We show that this metric is correlated with human judgments of topic quality as well as with the consistency of topics appearing across multiple models. At the word level, we experiment with different methods for adjusting individual word probabilities within topics based on their uncertainty. Humans prefer words ranked by our adjusted estimates nearly twice as often when compared to the traditional approach. Finally, we describe how the ideas presented in this work could potentially applied to other predictive or exploratory models in future work.

Narrow lenses for capturing the complexity of fisheries: A topic analysis of fisheries science from 1990 to 2016

AbstractDespite increased fisheries science output and publication outlets, the global crisis in fisheries management is as present as ever. Since a narrow research focus may be a contributing factor to this failure, this study uncovers topics in fisheries research and their trends over time. This interdisciplinary research evaluates whether science is diversifying fisheries research topics in an attempt to capture the complexity of the fisheries system, or whether it is multiplying research on similar topics, attempting to achieve an in‐depth, but possibly marginal, understanding of a few selected components of this system. By utilizing latent Dirichlet allocation as a generative probabilistic topic model, we analyse a unique dataset consisting of 46,582 full‐text articles published in the last 26 years in 21 specialized scientific fisheries journals. Among the 25 topics uncovered by the model, only one (Fisheries management) refers to the human dimension of fisheries understood as socio‐ecological complex adaptive systems. The most prevalent topics in our dataset directly relating to fisheries refer to Fisheries management, Stock assessment, and Fishing gear, with Fisheries management attracting the most interest. We propose directions for future research focus that most likely could contribute to providing useful advice for successful management of fisheries.

Co-occurrence of medical conditions: Exposing patterns through probabilistic topic modeling of snomed codes

Patients associated with multiple co-occurring health conditions often face aggravated complications and less favorable outcomes. Co-occurring conditions are especially prevalent among individuals suffering from kidney disease, an increasingly widespread condition affecting 13% of the general population in the US. This study aims to identify and characterize patterns of co-occurring medical conditions in patients employing a probabilistic framework. Specifically, we apply topic modeling in a non-traditional way to find associations across SNOMED-CT codes assigned and recorded in the EHRs of >13,000 patients diagnosed with kidney disease. Unlike most prior work on topic modeling, we apply the method to codes rather than to natural language. Moreover, we quantitatively evaluate the topics, assessing their tightness and distinctiveness, and also assess the medical validity of our results. Our experiments show that each topic is succinctly characterized by a few highly probable and unique disease codes, indicating that the topics are tight. Furthermore, inter-topic distance between each pair of topics is typically high, illustrating distinctiveness. Last, most coded conditions grouped together within a topic, are indeed reported to co-occur in the medical literature. Notably, our results uncover a few indirect associations among conditions that have hitherto not been reported as correlated in the medical literature.

You Are What You Eat: Exploring Rich Recipe Information for Cross-Region Food Analysis

Cuisine is a style of cooking and usually associated with a specific geographic region. Recipes from different cuisines shared on the web are an indicator of culinary cultures in different countries. Therefore, analysis of these recipes can lead to deep understanding of food from the cultural perspective. In this paper, we perform the first cross-region recipe analysis by jointly using the recipe ingredients, food images, and attributes such as the cuisine and course (e.g., main dish and dessert). For that solution, we propose a culinary culture analysis framework to discover the topics of ingredient bases and visualize them to enable various applications. We first propose a probabilistic topic model to discover cuisine-course specific topics. The manifold ranking method is then utilized to incorporate deep visual features to retrieve food images for topic visualization. At last, we applied the topic modeling and visualization method for three applications: 1) multimodal cuisine summarization with both recipe ingredients and images, 2) cuisine-course pattern analysis including topic-specific cuisine distribution and cuisine-specific course distribution of topics, and 3) cuisine recommendation for both cuisine-oriented and ingredient-oriented queries. Through these three applications, we can analyze the culinary cultures at both macro and micro levels. We conduct the experiment on a recipe database Yummly-66K with 66,615 recipes from 10 cuisines in Yummly. Qualitative and quantitative evaluation results have validated the effectiveness of topic modeling and visualization, and demonstrated the advantage of the framework in utilizing rich recipe information to analyze and interpret the culinary cultures from different regions.

Improving Topic Coherence Using Entity Extraction Denoising

AbstractManaging large collections of documents is an important problem for many areas of science, industry, and culture. Probabilistic topic modeling offers a promising solution. Topic modeling is an unsupervised machine learning method and the evaluation of this model is an interesting problem on its own. Topic interpretability measures have been developed in recent years as a more natural option for topic quality evaluation, emulating human perception of coherence with word sets correlation scores. In this paper, we show experimental evidence of the improvement of topic coherence score by restricting the training corpus to that of relevant information in the document obtained by Entity Recognition. We experiment with job advertisement data and find that with this approach topic models improve interpretability in about 40 percentage points on average. Our analysis reveals as well that using the extracted text chunks, some redundant topics are joined while others are split into moreskill-specifictopics. Fine-grained topics observed in models using the whole text are preserved.

A Web service search engine for large-scale Web service discovery based on the probabilistic topic modeling and clustering

With the ever increasing number of Web services, discovering an appropriate Web service requested by users has become a vital yet challenging task. We need a scalable and efficient search engine to deal with the large volume of Web services. The aim of this approach is to provide an efficient search engine that can retrieve the most relevant Web services in a short time. The proposed Web service search engine (WSSE) is based on the probabilistic topic modeling and clustering techniques that are integrated to support each other by discovering the semantic meaning of Web services and reducing the search space. The latent Dirichlet allocation (LDA) is used to extract topics from Web service descriptions. These topics are used to group similar Web services together. Each Web service description is represented as a topic vector, so the topic model is an efficient technique to reduce the dimensionality of word vectors and to discover the semantic meaning that is hidden in Web service descriptions. Also, the Web service description is represented as a word vector to address the drawbacks of the keyword-based search system. The accuracy of the proposed WSSE is compared with the keyword-based search system. Also, the precision and recall metrics are used to evaluate the performance of the proposed approach and the keyword-based search system. The results show that the proposed WSSE based on LDA and clustering outperforms the keyword-based search system.

Employee Satisfaction and Corporate Performance in the UK

This paper investigates the informational value of online reviews posted by employees for their employer, a rather untapped source of online information from employees, using a sample of 349,550 reviews from 40,915 UK firms. We explore this novel form of electronic Word-of-Mouth (e-WOM) from different perspectives, namely: (i) its information content as a tool to identify the drivers of job satisfaction/dissatisfaction, (ii) its predictive ability on firm financial performance and (iii) its operational and managerial value. Our approach considers both the rating score as well as the review text through a probabilistic topic modeling method, providing also a roadmap to quantify and exploit employee big data analytics. The novelty of this study lies in the coupling of structured and unstructured data for deriving managerial insights through a battery of econometric, financial and operational research methodologies. Our empirical analyses reveal that employee online reviews have informational value and incremental predictability gains for a firm’s internal and external stakeholders. The results indicate that when models integrate structured and unstructured big data there are leveraged opportunities for firms and managers to enhance the informativeness of decision support systems and in turn, gain competitive advantage.

Topic mining of tourist attractions based on a seasonal context aware LDA model

With the rise of personalized travel recommendation in recent years, automatic analysis and summary of the tourist attraction is of great importance in decision making for both tourists and tour operators. To this end, many probabilistic topic models have been proposed for feature extraction of tourist attraction. However, existing state-of-the-art probabilistic topic models overlook the fact that tourist attractions tend to have distinct characteristics with respect to specific seasonal context. In this article, we contribute the innovative idea of using seasonal contextual information to refine the characteristics of tourist attractions. Along this line, we first propose STLDA, a season topic model based on latent Dirichlet allocation which can capture meaningful topics corresponding to various seasonal contexts for each attraction. Then, an inference algorithm using Gibbs sampling is put forward to learn the model parameters of our proposed model. In order to verify the effectiveness of STLDA model, we present a detailed experimental study using collected real-world textual data of tourist attractions. The experimental analysis results show that the superiority of STLDA over the basic LDA model in providing a representative and comprehensive summarization related to each tourist attraction. More importantly, it has great significance for improving the level of personalized attraction recommendation.