Dynamic Topic Model Research Articles

Supporting secondary research in early drug discovery process through a Natural Language Processing based system

Abstract Last decades were characterised by a constant decline in the productivity of research and development activities of pharmaceutical companies. This is due to the fact that the drug discovery process contains an intrinsic risk that should be managed efficiently. Within this process, the early phase projects could be streamlined by doing more secondary research. These activities would involve the integration of chemical and biological knowledge from scientific literature in order to extract an overview and the evolution of a certain research area. This would then help refine the research and development operations. Considering the vast amount of pharmaceutical studies publications, it is not easy to identify the important information. For this task, a series of projects leveraged the advantages of the open pharmacological space through state-of-the-art technologies. The most popular are Knowledge Graphs methods. Although extremely useful, this technology requires increased investments of time and human resources. An alternative would be to develop a system that uses Natural Language Processing blocks. Still, there is no defined framework and reusable code template for the use-case of compounds development. In this study, it is presented the design and development of a system that uses Dynamic Topic Modelling and Named Entity Recognition modules in order to extract meaningful information from a large volume of unstructured texts. Moreover, the dynamic character of the topic modelling technique allows to analyse the evolution of different subject areas over time. In order to validate the system, a collection of articles from the Pharmaceutical Research Journal was used. Our results show that the system is able to identify the main research areas in the last 20 years, namely crystalline and amorphous systems, insulin resistance, paracellular permeability. Additionally, the evolution of the subjects is a highly valuable resource and should be used to get an in-depth understanding about the shifts that happened in a specific domain. However, a limitation of this system is that it cannot detect association between two concepts or entities if they are not involved in the same document.

The Prediction of Sales Volume and WoM Effect Based on Topic Extraction from Social Media

Purpose - This study attempts to predict the automobile sales volume and WoM Effect of electronic cars such as the Tesla Model S. Design/Methodology/Approach - This study extracted the data in a new industry, specifically, an electronic car such as the Tesla Model S based on the literature review of the existing media theory: the Media Richness Theory (MRT) and the Media Synchronicity Theory (MST). The unstructured data was used as text data for classifying important topics following the flow of time. Using Latent Dirichlet Allocation (LDA) and Dynamic Topic Modeling (DTM), we deducted the significant topic related to the company performance such as sales volume. Findings - We find out that the text mining approach allowed us to identify the derived indicators for an automobile manufacturer’s market success with centrality measures. Also, this research found evidence of conveyance and convergence for the communication process on the web relative to users’ web comments. Research Implications - Although previous studies used the economic approach, this study established that social media could capitalize on using conveyance and convergence of the communication process (e.g., conveyance vs. convergence) based on MST through theoretical consideration.

Investigation of Research Trends in the D(Data)ㆍ N(Network)ㆍA(A.I) Field Using the Dynamic Topic Model

Investigation of Research Trends in the D(Data)ㆍ N(Network)ㆍA(A.I) Field Using the Dynamic Topic Model

Collective Response to Media Coverage of the COVID-19 Pandemic on Reddit and Wikipedia: Mixed-Methods Analysis.

BackgroundThe exposure and consumption of information during epidemic outbreaks may alter people’s risk perception and trigger behavioral changes, which can ultimately affect the evolution of the disease. It is thus of utmost importance to map the dissemination of information by mainstream media outlets and the public response to this information. However, our understanding of this exposure-response dynamic during the COVID-19 pandemic is still limited.ObjectiveThe goal of this study is to characterize the media coverage and collective internet response to the COVID-19 pandemic in four countries: Italy, the United Kingdom, the United States, and Canada.MethodsWe collected a heterogeneous data set including 227,768 web-based news articles and 13,448 YouTube videos published by mainstream media outlets, 107,898 user posts and 3,829,309 comments on the social media platform Reddit, and 278,456,892 views of COVID-19–related Wikipedia pages. To analyze the relationship between media coverage, epidemic progression, and users’ collective web-based response, we considered a linear regression model that predicts the public response for each country given the amount of news exposure. We also applied topic modelling to the data set using nonnegative matrix factorization.ResultsOur results show that public attention, quantified as user activity on Reddit and active searches on Wikipedia pages, is mainly driven by media coverage; meanwhile, this activity declines rapidly while news exposure and COVID-19 incidence remain high. Furthermore, using an unsupervised, dynamic topic modeling approach, we show that while the levels of attention dedicated to different topics by media outlets and internet users are in good accordance, interesting deviations emerge in their temporal patterns.ConclusionsOverall, our findings offer an additional key to interpret public perception and response to the current global health emergency and raise questions about the effects of attention saturation on people’s collective awareness and risk perception and thus on their tendencies toward behavioral change.

A value of civic voices for smart city: A big data analysis of civic queries posed by Seoul citizens

Since urban problems are increasingly becoming complex and multifaceted due to the rapid urbanization, interest in the development of smart cities as an efficient solution to such problems is growing. The significance of smart cities lies in not having technology itself but using technology with a novel approach to solve urban problems, enhance the quality of life for urban residents, and optimize government performance. Such an approach includes scientifically processing civic query (including complaints, suggestions, and inquiry) data about a city and planning the city such that its policies reflect its residents' voices to ensure “throughput legitimacy.” This study introduces a novel approach to analyze diverse informal civic query data for a city and plan the city in a way that its residents want. By analyzing 160,000 civic queries accumulated over 10 years from 2006 to 2017 from the “Oasis of 10 Million Imagination,” which is a civic participation platform of Seoul, this study aims to contribute to the sustainable development of Seoul so that it can plan a citizen-centric and smart city that satisfies the demands of its residents. By applying Dynamic Topic Model, the authors intend to specify civic demands and forecast the demands of citizens.

Topic tracking model for analyzing student-generated posts in SPOC discussion forums

Due to an overwhelming amount of student-generated forum posts in small private online courses (SPOCs), students and instructors find it time-consuming and challenging to effectively navigate and track valuable information, such as the evolution of topics, emotional and behavioral changes in relation to topics. For solving this problem, this study analyzed plenty of discussion posts using an improved dynamic topic model, Time Information-Emotion Behavior Model (TI-EBTM). Time, emotion, and behavior characteristics were incorporated into the topic modeling process, which allowed for an overview of automatic tracking and understanding of temporal topic changes in SPOC discussion forums. The experiment on data from 30 SPOC courses showed that TI-EBTM outperformed other dynamic topic models and was effective in extracting prominent topics over time. Furthermore, we conducted an in-depth temporal topic analysis to investigate the utility of TI-EBTM in a case study. The results of the case study demonstrated that our methodology and analysis shed light on students’ temporal focuses (i.e., the changes of topic intensity and topic content) and reflected the evolution of topics’ emotional and behavioral tendencies. For example, students tended to express more negative emotions toward the topic about the method of data query by initiating the conversation at the end of the semester. The analytical results can provide instructors with valuable insights into the development of course forums and enable them to fine-tune course forums to suit students’ requirements, which will subsequently be helpful in enhancing discussion interaction and students’ learning experience.

Exploring Evolution of Public Opinions on Tianya Club Using Dynamic Topic Models

AbstractOnline media have brought tremendous changes to civic life, public opinions, and government administration. Compared with traditional media, online media not only allow individuals to browse news and express their views more freely, but also accelerate the transmission of opinions and expand influence. As public opinions may arouse societal unrest, it is worth detecting the primary topics and uncovering the evolution trends of public opinions for societal administration. Various algorithms are developed to deal with the huge volume of unstructured online media data. In this study, dynamic topic model is employed to explore topic content evolution and prevalence evolution using the original posts published from 2013 to 2017 on the Tianya Zatan Board of Tianya Club, which is one of the most popular BBS in China. Based on semantic similarities, topics are grouped into three themes: Family life, societal affairs, and government administration. The evolution of topic prevalence and content are affected by emergent incidents. Topics on family life become popular, while themes “societal affairs” and “government administration” with bigger standard deviations are more likely to be influenced by emergent hot events. Content evolution represented by monthly pairwise distance matrix is very easy to find change points of topic content.

Topic Evolution Analysis of COVID-19 News Articles

Recently, numerous media publishes various news on the latest developments every day due to the global spread of COVID-19. The news provides rich information about COVID-19 and includes a wide range of evolving topics. Our study is intended to develop a dynamic topic analysis system to monitor the evolution of the large-scale text data topics and assist with the social management and policymaking. The system expands the Dynamic Topic Model (DTM) with two modules: data sparsity computing and topic number selecting, which makes the experimental process more natural and generalizable. Data sparsity is designed to determine the length of single time slice. UCI, UMass and NPMI are introduced for choosing the optimal number of topics. This paper explores CBC news articles using DTM and captures the impact of COVID-19 on various aspects and the development of specific events. The experimental results demonstrate the effectiveness of our system for discovering and tracking the evolving topics. This system also plays an important role to improve the awareness of the public and serves as an analysis platform for government.

Dynamic Stakeholder-Associated Topic Modeling on Public Concerns in Megainfrastructure Projects: Case of Hong Kong–Zhuhai–Macao Bridge

AbstractPublic concerns are essential for megainfrastructure project success, meanwhile profoundly influencing stakeholder performance. To understand the dynamics of public concerns and the project...

Emerging Topics in Brexit Debate on Twitter Around the Deadlines

The present study is focused on the online debate relating to the Brexit process, three years and half since the historical referendum that has sanctioned the divide of the United Kingdom from the European Union. In our analysis we consider a corpus of approximately 33 million Brexit related tweets, shared on Twitter for 58 weeks, spanning from 31 December 2019 to 9 February 2020. Due to its great accessibility to data, Twitter constitutes a convenient data source to monitor and evaluate a wide variety of topics. In addition, Twitter’s marked orientation towards news and the dissemination of information makes this microblogging network more connected to politics compared to other platforms. Through static and dynamic topic modelling techniques, we were able to identify the topics that have attracted the most attention from Twitters users and to characterise their temporal evolution. The topics retrieved by the static model highlight the major events of the Brexit process while the dynamic analysis recovered the persistent themes of discussion and debate over the entire period.

동적토픽모델링과 네트워크 분석 기반의 기술융합 동향분석 : ICT-Agritech 융합 사례 연구

동적토픽모델링과 네트워크 분석 기반의 기술융합 동향분석 : ICT-Agritech 융합 사례 연구

GDTM: Graph-based Dynamic Topic Models

Dynamic Topic Modeling (DTM) is the ultimate solution for extracting topics from short texts generated in Online Social Networks (OSNs) like Twitter. It requires to be scalable and to be able to account for sparsity and dynamicity of short texts. Current solutions combine probabilistic mixture models like Dirichlet Multinomial or Pitman-Yor Process with approximate inference approaches like Gibbs Sampling and Stochastic Variational Inference to, respectively, account for dynamicity and scalability of DTM. However, these methods basically rely on weak probabilistic language models, which do not account for sparsity in short texts. In addition, their inference is based on iterative optimizations, which have scalability issues when it comes to DTM. We present GDTM, a single-pass graph-based DTM algorithm, to solve the problem. GDTM combines a context-rich and incremental feature representation method with graph partitioning to address scalability and dynamicity and uses a rich language model to account for sparsity. We run multiple experiments over a large-scale Twitter dataset to analyze the accuracy and scalability of GDTM and compare the results with four state-of-the-art models. In result, GDTM outperforms the best model by 11% on accuracy and performs by an order of magnitude faster while creating four times better topic quality over standard evaluation metrics.

What’s the talk in Brussels? Leveraging daily news coverage to measure issue attention in the European Union

Research on issue attention in the European Union has focused on the prominence of EU integration in domestic politics and media and, at EU level, on the salience of individual issues and legislative files, often in relation to lobbying. Existing EU-level measures of issue saliency, though, are limited in scope and periodicity and tend to reflect the policy priorities of a single institutional actor rather than that of the broader EU elite sphere. We present an alternative measure of issue attention leveraging the quasi-institutional nature of the Agence Europe daily bulletin which provides comprehensive but independent news coverage of EU affairs. We use text-mining techniques, including dynamic topic modelling, in combination with manual classification to map issue prevalence between 1979 and 2018. In addition to reporting validation results, we illustrate how our measure relates to other indicators of EU agenda formation and explain how researchers can make use of our new dataset.

Understanding Weekly COVID-19 Concerns through Dynamic Content-Specific LDA Topic Modeling.

The novelty and global scale of the COVID-19 pandemic has lead to rapid societal changes in a short span of time. As government policy and health measures shift, public perceptions and concerns also change, an evolution documented within discourse on social media. We propose a dynamic content-specific LDA topic modeling technique that can help to identify different domains of COVID-specific discourse that can be used to track societal shifts in concerns or views. Our experiments show that these model-derived topics are more coherent than standard LDA topics, and also provide new features that are more helpful in prediction of COVID-19 related outcomes including mobility and unemployment rate.

Adapting CRISP-DM for Idea Mining

Data mining project managers can benefit from using standard data mining process models. The benefits of using standard process models for data mining, such as the de facto and the most popular, Cross-Industry-Standard-Process model for Data Mining (CRISP-DM) are reduced cost and time. Also, standard models facilitate knowledge transfer, reuse of best practices, and minimize knowledge requirements. On the other hand, to unlock the potential of ever-growing textual data such as publications, patents, social media data, and documents of various forms, digital innovation is increasingly needed. Furthermore, the introduction of cutting-edge machine learning tools and techniques enable the elicitation of ideas. The processing of unstructured textual data to generate new and useful ideas is referred to as idea mining. Existing literature about idea mining merely overlooks the utilization of standard data mining process models. Therefore, the purpose of this paper is to propose a reusable model to generate ideas, CRISP-DM, for Idea Mining (CRISP-IM). The design and development of the CRISP-IM are done following the design science approach. The CRISP-IM facilitates idea generation, through the use of Dynamic Topic Modeling (DTM), unsupervised machine learning, and subsequent statistical analysis on a dataset of scholarly articles. The adapted CRISP-IM can be used to guide the process of identifying trends using scholarly literature datasets or temporally organized patent or any other textual dataset of any domain to elicit ideas. The ex-post evaluation of the CRISP-IM is left for future study.

Dynamic wavelet fingerprint for differentiation of tweet storm types

We describe a novel method for analyzing topics extracted from Twitter by utilizing the dynamic wavelet fingerprint technique (DWFT). Topics are derived from 7 different tweet storms analyzed in the study by using a dynamic topic model. Using the time series of each topic, we run DWFT analyses to get a two-dimensional, time-scale, binary image. Gaussian mixture model clustering is used to identify individual objects, or storm cells, that are characteristic to specific local behaviors commonly occurring in topics. The DWFT time series transformation is volume agnostic, meaning we can compare tweet storms of different intensities. We find that we can identify behavior, localized in time, that is characteristic to how different topics propagate through Twitter. The use of dynamic topic models and the DWFT create the basis for future applications as a real-time Twitter analysis system for flagging fake news.

Tracking urban geo-topics based on dynamic topic model

Modern cities are facing critical environmental and social problems that are difficult to solve using conventional planning approaches due to the cities’ magnitude and complexity. Recent developments in sensing technologies and urban computing, however, integrate new data resources and technologies to tackle these challenges. Popular social networking platforms such as Twitter provide new data sources on important events (e.g., cultural activities, political campaigns, accidents, crises) providing rich knowledge about urban systems and human dynamics. This research is intended to develop a method for effectively monitoring important information during such events and helping with planning and policymaking. We use semantically similar and geographically close geo-topics to represent important local events. This research proposes a data-driven system for detecting and tracking the semantic, spatial, and temporal dynamics of these geo-topics, specifically designed for geo-tagged tweets. The system consists of data preprocessing, geo-topic generation, and geo-topic tracking modules. The preprocessing module can remove robotic and semantically trivial texts. In the geo-topic generation module, we use spatial factors to measure the spatial impacts of geo-tagged tweets by applying an exponential decay function to the pairwise distances between tweets. We then improve the dynamic topic model (DTM) by embedding the spatial factors to enable the generation of geo-topics in semantic, spatial, and temporal dimensions simultaneously. The geo-topic tracking module monitors semantic change by detecting changes in certain keywords’ probabilities and the volumes of tweets belonging to different geo-topics. This module also uses radius of gyration and trajectory-pattern mining to track and analyze the movement patterns of geo-topics. We employed the tracking system in three disaster cases in different U.S. cities to track small-scale emergencies and crises. These implementations demonstrated the effectiveness of the system for identifying and tracking geo-topics at fine temporal and geographic scales. The system also has strong potential in creating planning-related analyses for policy makers, improving the situational awareness of the general public, and serving as a basis for urban information systems that contribute to smart, agile, and resilient city developments.

메타데이터를 활용한 1960∼2018 <한국언론학보> 논문 분석 : 다이내믹 토픽 모델링(Dynamic Topic Modeling) 방법을 중심으로

이 연구는 〈한국언론학보〉 창간호부터 2018년까지 논문 전집 메타데이터를 활용하여 두 가지 접근으로 학보의 60년을 분석하였다. 우선 기술통계 분석 결과 〈한국언론학보〉의 총 논문 수는 2,048편, 학보논문 저자는 1,276명으로 나타났다. 2001년 이후 누적 논문 수는 2000년 이전과 비교할 때 양적으로 약 3배 증가하였으나 최근 4년간 논문 수가 급감해 학보의 역할에 대한 논의가 필요함을 제안하였다. 학보 논문의 주요 토픽이 60년간 어떠한 진화 패턴으로 나타나는지 알아보기 위해 시계열 데이터 연산 알고리즘인 다이내믹 토픽 모델링(DTM : Dynamic Topic Modeling)으로 분석한 결과 첫째, 연구영역에서는 저널리즘과 온라인 저널리즘, 보도관련 저널리즘이 발생확률이 높은 토픽으로 추출된 반면, 문화연구는 발생확률이 낮게 나타났다. 둘째, 방법론에서는 경험주의 연구와 실증연구가 지배적 토픽으로 나타났고 컴퓨테이셔널 방법론도 증가추세로 나타났다. 셋째, 이론에서는 프레임, 이용과 충족, 의제설정이 주요 토픽으로 추출되었으며 넷째, 메시지연구는 효과 관련 토픽이, 다섯째, 수용자 연구에서는 정치참여, 텔레비전 시청, 청소년 관련 토픽이 지배적으로 나타났다. 마지막으로 연구대상에서는 온라인, 소셜미디어, 인터넷 토론 관련 토픽이 지배적으로 나타나 온라인 관련 연구가 활성화되었음을 알 수 있었다. 이 연구는 내용분석 분류 방법과 달리 메타데이터를 활용해 알고리즘과 기계학습으로 토픽을 추출하고 이를 군집화(clustering)하여 학보 논문의 연구경향 분석방법을 새롭게 제안해 논의했다는데 의의가 있다.

Adaptive Model for Dynamic and Temporal Topic Modeling from Big Data using Deep Learning Architecture

Adaptive Model for Dynamic and Temporal Topic Modeling from Big Data using Deep Learning Architecture

Technological Innovations in Disease Management: Text Mining US Patent Data From 1995 to 2017.

BackgroundPatents are important intellectual property protecting technological innovations that inspire efficient research and development in biomedicine. The number of awarded patents serves as an important indicator of economic growth and technological innovation. Researchers have mined patents to characterize the focuses and trends of technological innovations in many fields.ObjectiveTo expand patent mining to biomedicine and facilitate future resource allocation in biomedical research for the United States, we analyzed US patent documents to determine the focuses and trends of protected technological innovations across the entire disease landscape.MethodsWe analyzed more than 5 million US patent documents between 1995 and 2017, using summary statistics and dynamic topic modeling. More specifically, we investigated the disease coverage and latent topics in patent documents over time. We also incorporated the patent data into the calculation of our recently developed Research Opportunity Index (ROI) and Public Health Index (PHI), to recalibrate the resource allocation in biomedical research.ResultsOur analysis showed that protected technological innovations have been primarily focused on socioeconomically critical diseases such as “other cancers” (malignant neoplasm of head, face, neck, abdomen, pelvis, or limb; disseminated malignant neoplasm; Merkel cell carcinoma; and malignant neoplasm, malignant carcinoid tumors, neuroendocrine tumor, and carcinoma in situ of an unspecified site), diabetes mellitus, and obesity. The United States has significantly improved resource allocation to biomedical research and development over the past 17 years, as illustrated by the decreasing PHI. Diseases with positive ROI, such as ankle and foot fracture, indicate potential research opportunities for the future. Development of novel chemical or biological drugs and electrical devices for diagnosis and disease management is the dominating topic in patented inventions.ConclusionsThis multifaceted analysis of patent documents provides a deep understanding of the focuses and trends of technological innovations in disease management in patents. Our findings offer insights into future research and innovation opportunities and provide actionable information to facilitate policy makers, payers, and investors to make better evidence-based decisions regarding resource allocation in biomedicine.