Dynamic Topic Model Research Articles

A value of civic voices for smart city: A big data analysis of civic queries posed by Seoul citizens

Since urban problems are increasingly becoming complex and multifaceted due to the rapid urbanization, interest in the development of smart cities as an efficient solution to such problems is growing. The significance of smart cities lies in not having technology itself but using technology with a novel approach to solve urban problems, enhance the quality of life for urban residents, and optimize government performance. Such an approach includes scientifically processing civic query (including complaints, suggestions, and inquiry) data about a city and planning the city such that its policies reflect its residents' voices to ensure “throughput legitimacy.” This study introduces a novel approach to analyze diverse informal civic query data for a city and plan the city in a way that its residents want. By analyzing 160,000 civic queries accumulated over 10 years from 2006 to 2017 from the “Oasis of 10 Million Imagination,” which is a civic participation platform of Seoul, this study aims to contribute to the sustainable development of Seoul so that it can plan a citizen-centric and smart city that satisfies the demands of its residents. By applying Dynamic Topic Model, the authors intend to specify civic demands and forecast the demands of citizens.

Topic tracking model for analyzing student-generated posts in SPOC discussion forums

Due to an overwhelming amount of student-generated forum posts in small private online courses (SPOCs), students and instructors find it time-consuming and challenging to effectively navigate and track valuable information, such as the evolution of topics, emotional and behavioral changes in relation to topics. For solving this problem, this study analyzed plenty of discussion posts using an improved dynamic topic model, Time Information-Emotion Behavior Model (TI-EBTM). Time, emotion, and behavior characteristics were incorporated into the topic modeling process, which allowed for an overview of automatic tracking and understanding of temporal topic changes in SPOC discussion forums. The experiment on data from 30 SPOC courses showed that TI-EBTM outperformed other dynamic topic models and was effective in extracting prominent topics over time. Furthermore, we conducted an in-depth temporal topic analysis to investigate the utility of TI-EBTM in a case study. The results of the case study demonstrated that our methodology and analysis shed light on students’ temporal focuses (i.e., the changes of topic intensity and topic content) and reflected the evolution of topics’ emotional and behavioral tendencies. For example, students tended to express more negative emotions toward the topic about the method of data query by initiating the conversation at the end of the semester. The analytical results can provide instructors with valuable insights into the development of course forums and enable them to fine-tune course forums to suit students’ requirements, which will subsequently be helpful in enhancing discussion interaction and students’ learning experience.

Exploring Evolution of Public Opinions on Tianya Club Using Dynamic Topic Models

AbstractOnline media have brought tremendous changes to civic life, public opinions, and government administration. Compared with traditional media, online media not only allow individuals to browse news and express their views more freely, but also accelerate the transmission of opinions and expand influence. As public opinions may arouse societal unrest, it is worth detecting the primary topics and uncovering the evolution trends of public opinions for societal administration. Various algorithms are developed to deal with the huge volume of unstructured online media data. In this study, dynamic topic model is employed to explore topic content evolution and prevalence evolution using the original posts published from 2013 to 2017 on the Tianya Zatan Board of Tianya Club, which is one of the most popular BBS in China. Based on semantic similarities, topics are grouped into three themes: Family life, societal affairs, and government administration. The evolution of topic prevalence and content are affected by emergent incidents. Topics on family life become popular, while themes “societal affairs” and “government administration” with bigger standard deviations are more likely to be influenced by emergent hot events. Content evolution represented by monthly pairwise distance matrix is very easy to find change points of topic content.

Topic Evolution Analysis of COVID-19 News Articles

Recently, numerous media publishes various news on the latest developments every day due to the global spread of COVID-19. The news provides rich information about COVID-19 and includes a wide range of evolving topics. Our study is intended to develop a dynamic topic analysis system to monitor the evolution of the large-scale text data topics and assist with the social management and policymaking. The system expands the Dynamic Topic Model (DTM) with two modules: data sparsity computing and topic number selecting, which makes the experimental process more natural and generalizable. Data sparsity is designed to determine the length of single time slice. UCI, UMass and NPMI are introduced for choosing the optimal number of topics. This paper explores CBC news articles using DTM and captures the impact of COVID-19 on various aspects and the development of specific events. The experimental results demonstrate the effectiveness of our system for discovering and tracking the evolving topics. This system also plays an important role to improve the awareness of the public and serves as an analysis platform for government.

Dynamic Stakeholder-Associated Topic Modeling on Public Concerns in Megainfrastructure Projects: Case of Hong Kong–Zhuhai–Macao Bridge

AbstractPublic concerns are essential for megainfrastructure project success, meanwhile profoundly influencing stakeholder performance. To understand the dynamics of public concerns and the project...

Emerging Topics in Brexit Debate on Twitter Around the Deadlines

The present study is focused on the online debate relating to the Brexit process, three years and half since the historical referendum that has sanctioned the divide of the United Kingdom from the European Union. In our analysis we consider a corpus of approximately 33 million Brexit related tweets, shared on Twitter for 58 weeks, spanning from 31 December 2019 to 9 February 2020. Due to its great accessibility to data, Twitter constitutes a convenient data source to monitor and evaluate a wide variety of topics. In addition, Twitter’s marked orientation towards news and the dissemination of information makes this microblogging network more connected to politics compared to other platforms. Through static and dynamic topic modelling techniques, we were able to identify the topics that have attracted the most attention from Twitters users and to characterise their temporal evolution. The topics retrieved by the static model highlight the major events of the Brexit process while the dynamic analysis recovered the persistent themes of discussion and debate over the entire period.

동적토픽모델링과 네트워크 분석 기반의 기술융합 동향분석 : ICT-Agritech 융합 사례 연구

동적토픽모델링과 네트워크 분석 기반의 기술융합 동향분석 : ICT-Agritech 융합 사례 연구

GDTM: Graph-based Dynamic Topic Models

Dynamic Topic Modeling (DTM) is the ultimate solution for extracting topics from short texts generated in Online Social Networks (OSNs) like Twitter. It requires to be scalable and to be able to account for sparsity and dynamicity of short texts. Current solutions combine probabilistic mixture models like Dirichlet Multinomial or Pitman-Yor Process with approximate inference approaches like Gibbs Sampling and Stochastic Variational Inference to, respectively, account for dynamicity and scalability of DTM. However, these methods basically rely on weak probabilistic language models, which do not account for sparsity in short texts. In addition, their inference is based on iterative optimizations, which have scalability issues when it comes to DTM. We present GDTM, a single-pass graph-based DTM algorithm, to solve the problem. GDTM combines a context-rich and incremental feature representation method with graph partitioning to address scalability and dynamicity and uses a rich language model to account for sparsity. We run multiple experiments over a large-scale Twitter dataset to analyze the accuracy and scalability of GDTM and compare the results with four state-of-the-art models. In result, GDTM outperforms the best model by 11% on accuracy and performs by an order of magnitude faster while creating four times better topic quality over standard evaluation metrics.

What’s the talk in Brussels? Leveraging daily news coverage to measure issue attention in the European Union

Research on issue attention in the European Union has focused on the prominence of EU integration in domestic politics and media and, at EU level, on the salience of individual issues and legislative files, often in relation to lobbying. Existing EU-level measures of issue saliency, though, are limited in scope and periodicity and tend to reflect the policy priorities of a single institutional actor rather than that of the broader EU elite sphere. We present an alternative measure of issue attention leveraging the quasi-institutional nature of the Agence Europe daily bulletin which provides comprehensive but independent news coverage of EU affairs. We use text-mining techniques, including dynamic topic modelling, in combination with manual classification to map issue prevalence between 1979 and 2018. In addition to reporting validation results, we illustrate how our measure relates to other indicators of EU agenda formation and explain how researchers can make use of our new dataset.

Understanding Weekly COVID-19 Concerns through Dynamic Content-Specific LDA Topic Modeling.

The novelty and global scale of the COVID-19 pandemic has lead to rapid societal changes in a short span of time. As government policy and health measures shift, public perceptions and concerns also change, an evolution documented within discourse on social media. We propose a dynamic content-specific LDA topic modeling technique that can help to identify different domains of COVID-specific discourse that can be used to track societal shifts in concerns or views. Our experiments show that these model-derived topics are more coherent than standard LDA topics, and also provide new features that are more helpful in prediction of COVID-19 related outcomes including mobility and unemployment rate.

Adapting CRISP-DM for Idea Mining

Data mining project managers can benefit from using standard data mining process models. The benefits of using standard process models for data mining, such as the de facto and the most popular, Cross-Industry-Standard-Process model for Data Mining (CRISP-DM) are reduced cost and time. Also, standard models facilitate knowledge transfer, reuse of best practices, and minimize knowledge requirements. On the other hand, to unlock the potential of ever-growing textual data such as publications, patents, social media data, and documents of various forms, digital innovation is increasingly needed. Furthermore, the introduction of cutting-edge machine learning tools and techniques enable the elicitation of ideas. The processing of unstructured textual data to generate new and useful ideas is referred to as idea mining. Existing literature about idea mining merely overlooks the utilization of standard data mining process models. Therefore, the purpose of this paper is to propose a reusable model to generate ideas, CRISP-DM, for Idea Mining (CRISP-IM). The design and development of the CRISP-IM are done following the design science approach. The CRISP-IM facilitates idea generation, through the use of Dynamic Topic Modeling (DTM), unsupervised machine learning, and subsequent statistical analysis on a dataset of scholarly articles. The adapted CRISP-IM can be used to guide the process of identifying trends using scholarly literature datasets or temporally organized patent or any other textual dataset of any domain to elicit ideas. The ex-post evaluation of the CRISP-IM is left for future study.

Dynamic wavelet fingerprint for differentiation of tweet storm types

We describe a novel method for analyzing topics extracted from Twitter by utilizing the dynamic wavelet fingerprint technique (DWFT). Topics are derived from 7 different tweet storms analyzed in the study by using a dynamic topic model. Using the time series of each topic, we run DWFT analyses to get a two-dimensional, time-scale, binary image. Gaussian mixture model clustering is used to identify individual objects, or storm cells, that are characteristic to specific local behaviors commonly occurring in topics. The DWFT time series transformation is volume agnostic, meaning we can compare tweet storms of different intensities. We find that we can identify behavior, localized in time, that is characteristic to how different topics propagate through Twitter. The use of dynamic topic models and the DWFT create the basis for future applications as a real-time Twitter analysis system for flagging fake news.

Tracking urban geo-topics based on dynamic topic model

Modern cities are facing critical environmental and social problems that are difficult to solve using conventional planning approaches due to the cities’ magnitude and complexity. Recent developments in sensing technologies and urban computing, however, integrate new data resources and technologies to tackle these challenges. Popular social networking platforms such as Twitter provide new data sources on important events (e.g., cultural activities, political campaigns, accidents, crises) providing rich knowledge about urban systems and human dynamics. This research is intended to develop a method for effectively monitoring important information during such events and helping with planning and policymaking. We use semantically similar and geographically close geo-topics to represent important local events. This research proposes a data-driven system for detecting and tracking the semantic, spatial, and temporal dynamics of these geo-topics, specifically designed for geo-tagged tweets. The system consists of data preprocessing, geo-topic generation, and geo-topic tracking modules. The preprocessing module can remove robotic and semantically trivial texts. In the geo-topic generation module, we use spatial factors to measure the spatial impacts of geo-tagged tweets by applying an exponential decay function to the pairwise distances between tweets. We then improve the dynamic topic model (DTM) by embedding the spatial factors to enable the generation of geo-topics in semantic, spatial, and temporal dimensions simultaneously. The geo-topic tracking module monitors semantic change by detecting changes in certain keywords’ probabilities and the volumes of tweets belonging to different geo-topics. This module also uses radius of gyration and trajectory-pattern mining to track and analyze the movement patterns of geo-topics. We employed the tracking system in three disaster cases in different U.S. cities to track small-scale emergencies and crises. These implementations demonstrated the effectiveness of the system for identifying and tracking geo-topics at fine temporal and geographic scales. The system also has strong potential in creating planning-related analyses for policy makers, improving the situational awareness of the general public, and serving as a basis for urban information systems that contribute to smart, agile, and resilient city developments.

메타데이터를 활용한 1960∼2018 <한국언론학보> 논문 분석 : 다이내믹 토픽 모델링(Dynamic Topic Modeling) 방법을 중심으로

이 연구는 〈한국언론학보〉 창간호부터 2018년까지 논문 전집 메타데이터를 활용하여 두 가지 접근으로 학보의 60년을 분석하였다. 우선 기술통계 분석 결과 〈한국언론학보〉의 총 논문 수는 2,048편, 학보논문 저자는 1,276명으로 나타났다. 2001년 이후 누적 논문 수는 2000년 이전과 비교할 때 양적으로 약 3배 증가하였으나 최근 4년간 논문 수가 급감해 학보의 역할에 대한 논의가 필요함을 제안하였다. 학보 논문의 주요 토픽이 60년간 어떠한 진화 패턴으로 나타나는지 알아보기 위해 시계열 데이터 연산 알고리즘인 다이내믹 토픽 모델링(DTM : Dynamic Topic Modeling)으로 분석한 결과 첫째, 연구영역에서는 저널리즘과 온라인 저널리즘, 보도관련 저널리즘이 발생확률이 높은 토픽으로 추출된 반면, 문화연구는 발생확률이 낮게 나타났다. 둘째, 방법론에서는 경험주의 연구와 실증연구가 지배적 토픽으로 나타났고 컴퓨테이셔널 방법론도 증가추세로 나타났다. 셋째, 이론에서는 프레임, 이용과 충족, 의제설정이 주요 토픽으로 추출되었으며 넷째, 메시지연구는 효과 관련 토픽이, 다섯째, 수용자 연구에서는 정치참여, 텔레비전 시청, 청소년 관련 토픽이 지배적으로 나타났다. 마지막으로 연구대상에서는 온라인, 소셜미디어, 인터넷 토론 관련 토픽이 지배적으로 나타나 온라인 관련 연구가 활성화되었음을 알 수 있었다. 이 연구는 내용분석 분류 방법과 달리 메타데이터를 활용해 알고리즘과 기계학습으로 토픽을 추출하고 이를 군집화(clustering)하여 학보 논문의 연구경향 분석방법을 새롭게 제안해 논의했다는데 의의가 있다.

Adaptive Model for Dynamic and Temporal Topic Modeling from Big Data using Deep Learning Architecture

Adaptive Model for Dynamic and Temporal Topic Modeling from Big Data using Deep Learning Architecture

Technological Innovations in Disease Management: Text Mining US Patent Data From 1995 to 2017.

BackgroundPatents are important intellectual property protecting technological innovations that inspire efficient research and development in biomedicine. The number of awarded patents serves as an important indicator of economic growth and technological innovation. Researchers have mined patents to characterize the focuses and trends of technological innovations in many fields.ObjectiveTo expand patent mining to biomedicine and facilitate future resource allocation in biomedical research for the United States, we analyzed US patent documents to determine the focuses and trends of protected technological innovations across the entire disease landscape.MethodsWe analyzed more than 5 million US patent documents between 1995 and 2017, using summary statistics and dynamic topic modeling. More specifically, we investigated the disease coverage and latent topics in patent documents over time. We also incorporated the patent data into the calculation of our recently developed Research Opportunity Index (ROI) and Public Health Index (PHI), to recalibrate the resource allocation in biomedical research.ResultsOur analysis showed that protected technological innovations have been primarily focused on socioeconomically critical diseases such as “other cancers” (malignant neoplasm of head, face, neck, abdomen, pelvis, or limb; disseminated malignant neoplasm; Merkel cell carcinoma; and malignant neoplasm, malignant carcinoid tumors, neuroendocrine tumor, and carcinoma in situ of an unspecified site), diabetes mellitus, and obesity. The United States has significantly improved resource allocation to biomedical research and development over the past 17 years, as illustrated by the decreasing PHI. Diseases with positive ROI, such as ankle and foot fracture, indicate potential research opportunities for the future. Development of novel chemical or biological drugs and electrical devices for diagnosis and disease management is the dominating topic in patented inventions.ConclusionsThis multifaceted analysis of patent documents provides a deep understanding of the focuses and trends of technological innovations in disease management in patents. Our findings offer insights into future research and innovation opportunities and provide actionable information to facilitate policy makers, payers, and investors to make better evidence-based decisions regarding resource allocation in biomedicine.

GDTM: A Gaussian Dynamic Topic Model for Forwarding Prediction Under Complex Mechanisms

Forwarding is the keyway for information propagation in social networks that affected by complex factors. Due to the fact that the forwarding mechanisms are essentially unclear, this paper focuses on the formation and evolution of the external as well as internal driving mechanisms and develops a Gaussian dynamic topic model for forwarding prediction by incorporating all the information of nodes and edges. First, based on the diversity of communities each user located in, latent Dirichlet allocation (LDA) traditional text modeling method is applied to user following relationships modeling that leads to the initial formation of communities, and user interacting relationships modeling that leads to the evolution of communities. Taking the advantage of LDA topic model in dealing with the problem of polysemy and synonym, we can mine user latent distribution over communities and analyze the external community driving effects. Second, considering the differences in individual habits, time factor is introduced and the dynamic topic model is proposed to model user behavioral attributes. Meanwhile, in regard to continuous user attributes modeling, the parameter of topic-word distribution in the topic model is replaced by multivariate Gaussian distributions, which shown to be effective at capturing the regularities of individual behavioral habits and analyzing the internal individual driving effects. Finally, combining with external and internal factors, a probabilistic graph model is used to modeling forwarding behavior. We propose methods based on Gibbs sampling and an expectation–maximization algorithm to estimate model parameters and fit our model to predict user forwarding actions. Experimental results indicate that the model can not only detect the latent communities but also can improve the performance of forwarding prediction effectively.

Bayesian Analysis of Dynamic Linear Topic Models

Discovering temporal evolution of themes from a time-stamped collection of text poses a challenging statistical learning problem. Dynamic topic models offer a probabilistic modeling framework to decompose a corpus of text documents into “topics”, i.e., probability distributions over vocabulary terms, while simultaneously learning the temporal dynamics of the relative prevalence of these topics. We extend the dynamic topic model of Blei and Lafferty (2006) by fusing its multinomial factor model on topics with dynamic linear models that account for time trends and seasonality in topic prevalence. A Markov chain Monte Carlo (MCMC) algorithm that utilizes Pólya-Gamma data augmentation is developed for posterior sampling. Conditional independencies in the model and sampling are made explicit, and our MCMC algorithm is parallelized where possible to allow for inference in large corpora. Our model and inference algorithm are validated with multiple synthetic examples, and we consider the applied problem of modeling trends in real estate listings from the housing website Zillow. We demonstrate in synthetic examples that sharing information across documents is critical for accurately estimating document-specific topic proportions. Analysis of the Zillow corpus demonstrates that the method is able to learn seasonal patterns and locally linear trends in topic prevalence.

Discovering author interest evolution in order-sensitive and Semantic-aware topic modeling

Modeling the interests of authors over time from documents has important applications in broad applications such as recommendation systems, authorship identification and opinion extraction. In this paper, we propose an Ordering-sensitive and Semantic-aware Dynamic Author Topic Model (OSDATM), which monitors the evolution of author interest in time-stamped documents. The model further uses the discovered author interest information to discover better topics. Unlike traditional topic models, OSDATM is sensitive to the ordering of words, thus it extracts more information from the semantic meaning of the context. The experimental results show that OSDATM learns better topics than state-of-the-art topic models. In addition, the dynamic interests of authors that the OSDATM model discovers are interpretable and consistent with the truth.

Event mining and timeliness analysis from heterogeneous news streams

News documents published online represent an important source of information that can be used for event detection and tracking as well as for analyzing the temporal publishing relationships among different news streams.In this paper, we describe our research on detecting, tracking, and predicting events from multiple news streams. We also analyze the temporal publishing patterns of newswires on different platforms and their timeliness in reporting the events. First, we present an approach based on discrete dynamic topic modeling and Hidden Markov Model for event detection and tracking. Then, we predict the events that would persist in the next time slice, which can be important for forecasting facts that would be popular in the future. We leverage the detected events for clustering news documents according to the events they describe. This allows us to determine which newswires published news about an event and to analyze their temporal ordering in reporting events. Finally, we propose two scoring functions for ranking the newswires based on their timeliness.We tested our methodologies on different collections of news articles and tweets. Moreover, we built a collection of heterogeneous news documents with event-document labels which were manually assessed using crowdsourcing. Experimental results showed that, compared to the traditional dynamic topic model, our approach is able to timely detect emerging topics (events). Overall, we could register an event coverage of about 90% w.r.t. the pool of labeled events. The evolution of events is captured by event chains which are highly coherent (0.76) and informative (0.60) allowing to effectively reconstruct the stories. Furthermore, the event-based clustering of news documents has a good trade-off of precision and recall (F-score = 0.83) and the topic keywords provide a semantic description of the events represented by the clusters. Concerning our analysis on the temporal publishing relationships among news streams, we could observe interesting patterns on the usage of the different platforms, for example, some newswires still favor their own official websites, while others tend to publish more timely on Twitter.