Dynamic Topic Model Research Articles

Sino-Japanese Relations from the Perspective of Chinese Mainstream Media During the Pandemic: Based on Word2vec and DTM Model

We conducted sentiment classification for news reports during the COVID-19 pandemic on Sino-Japanese relations from Chinese and Japanese media. Based on the Word2vec and Dynamic Topic Models, we analyzed the macroscopic dimension of theme distribution and the semantic change process of “Sino-Japanese” related words. The results indicate that Chinese media reports are mainly neutral, but Japanese ones are mainly negative. The meaning of the terms related to “Sino-Japanese” has not changed significantly, but there are minor differences in the content of reports on the same subject in different periods. During Suga’s administration, the overall stability of Sino-Japanese relations and the continued advancement of pragmatic cooperation are expected to continue.

Tracking patients healthcare experiences during the COVID-19 outbreak: Topic modeling and sentiment analysis of doctor reviews

Emerging voices of patients in the form of opinions and expectations about the quality of care can improve healthcare service quality. A large volume of patients’ opinions as online doctor reviews (ODRs) are available online to access, analyze, and improve patients’ perceptions. This paper aims to explore COVID-19-related conversations, complaints, and sentiments using ODRs posted by users of the physician rating website. We analyzed 96,234 ODRs of 5,621 physicians from a prominent health rating website in the United Kingdom (Iwantgreatcare.org) in threetime slices (i.e., from February 01 to October 31, 2020). We employed machine learning approach, dynamic topic modeling, to identify prominent bigrams, salient topics and labels, sentiments embedded in reviews and topics, and patient-perceived root cause and strengths, weaknesses, opportunities, and threats (SWOT) analyses to examine SWOT for healthcare organizations. This method finds a total of 30 latent topics with 10 topics across each time slice. The current study identified new discussion topics about COVID-19 occurring from time slice 1 to time slice 3, such as news about the COVID-19 pandemic, violence against the lockdown, quarantine process and quarantine centers at different locations, and vaccine development/treatment to stop virus spread. Sentiment analysis reveals that fear for novel pathogen prevails across all topics. Based on the SWOT analysis, our findings provide a clue for doctors, hospitals, and government officials to enhance patients’ satisfaction and minimize dissatisfaction by satisfying their needs and improve the quality of care during the COVID-19 crisis.

Topic evolution analysis of radar research using a dynamic topic model based on latent Dirichlet allocation

This work applies a dynamic topic model based on latent Dirichlet allocation to investigate the evolution of radar research and applications. To address the problem that processing a large number of papers exceeds the capacity of computers, we propose a method to sample documents according to their citation count. Of 104,428 sampled articles, 108 research topics are extracted. The evolution of topics is analysed from two dimensions: content and strength. The results show that radar technology, which arose mainly from physics and engineering science, has been widely applied in studies in the civil engineering, geographical, environmental, meteorological, geological, agricultural, ecological, among others. In the long-term development of radar, new technologies have continuously been produced. At the individual topic level, the research content has changed over time. The development objectives of radar systems are to enhance functionality, extract more information and improve clarity.

Recurrent Coupled Topic Modeling over Sequential Documents

The abundant sequential documents such as online archival, social media, and news feeds are streamingly updated, where each chunk of documents is incorporated with smoothly evolving yet dependent topics. Such digital texts have attracted extensive research on dynamic topic modeling to infer hidden evolving topics and their temporal dependencies. However, most of the existing approaches focus on single-topic-thread evolution and ignore the fact that a current topic may be coupled with multiple relevant prior topics. In addition, these approaches also incur the intractable inference problem when inferring latent parameters, resulting in a high computational cost and performance degradation. In this work, we assume that a current topic evolves from all prior topics with corresponding coupling weights, forming the multi-topic-thread evolution . Our method models the dependencies between evolving topics and thoroughly encodes their complex multi-couplings across time steps. To conquer the intractable inference challenge, a new solution with a set of novel data augmentation techniques is proposed, which successfully discomposes the multi-couplings between evolving topics. A fully conjugate model is thus obtained to guarantee the effectiveness and efficiency of the inference technique. A novel Gibbs sampler with a backward–forward filter algorithm efficiently learns latent time-evolving parameters in a closed-form. In addition, the latent Indian Buffet Process compound distribution is exploited to automatically infer the overall topic number and customize the sparse topic proportions for each sequential document without bias. The proposed method is evaluated on both synthetic and real-world datasets against the competitive baselines, demonstrating its superiority over the baselines in terms of the low per-word perplexity, high coherent topics, and better document time prediction.

Discovering the evolution of artificial intelligence in cancer research using dynamic topic modeling

The rapid growth of healthcare data in recent years calls for more advanced and efficient analytic techniques. Artificial intelligence facilitates finding insightful patterns in massive high-dimensional data. Considering the latest movements towards using machine learning and deep learning techniques in the medical domain, in this study, we focused on the publications in which researchers employed artificial intelligence techniques for cancer diagnosis and treatment. Using dynamic topic modeling and natural language processing techniques, we analyzed the contents and trends of more than 12,000 scientific publications within the period of 2000 to 2018, extracted from two different sources, i.e., Elsevier’s Scopus and PubMed. While drawing the landscape of cancer research, our results also shed light on the evolution of artificial intelligence techniques and algorithms used for cancer diagnosis and treatment. Our findings confirm that modern computer science algorithms are being widely applied to extract patterns from large-scale medical images to cure different types of cancer with a special focus on deep learning techniques in recent years.

Tracking and analysis of discourse dynamics and polarity during the early Corona pandemic in Iran

It has not been long since a new disease called COVID-19 has hit the international community. Unknown nature of the virus, evidence of its adaptability and survival in new conditions, its widespread prevalence and also lengthy recovery period, along with daily notifications of new infection and fatality statistics, have created a wave of fear and anxiety among the public community and authorities. These factors have led to extreme changes in the social discourse in a rather short period of time. The analysis of this discourse is important to reconcile the society and restore ordinary conditions of mental peace and health. Although much research has been done on the disease since its international pandemic, the sociological analysis of the recent public phenomenon, especially in developing countries, still needs attention. We propose a framework for analyzing social media data and news stories oriented around COVID-19 disease. Our research is based on an extensive Persian data set gathered from different social media networks and news agencies in the period of January 21-April 29, 2020. We use the Latent Dirichlet Allocation (LDA) model and dynamic topic modeling to understand and capture the change of discourse in terms of temporal subjects. We scrutinize the reasons of subject alternations by exploring the related events and adopted practices and policies. The social discourse can highly affect the community morale and polarization. Therefore, we further analyze the polarization in online social media posts, and detect points of concept drift in the stream. Based on the analyzed content, effective guidelines are extracted to shift polarization towards positive. The results show that the proposed framework is able to provide an effective practical approach for cause and effect analysis of the social discourse.

Understanding Cybersecurity Threat Trends Through Dynamic Topic Modeling.

Cybersecurity threats continue to increase and are impacting almost all aspects of modern life. Being aware of how vulnerabilities and their exploits are changing gives helpful insights into combating new threats. Applying dynamic topic modeling to a time-stamped cybersecurity document collection shows how the significance and details of concepts found in them are evolving. We correlate two different temporal corpora, one with reports about specific exploits and the other with research-oriented papers on cybersecurity vulnerabilities and threats. We represent the documents, concepts, and dynamic topic modeling data in a semantic knowledge graph to support integration, inference, and discovery. A critical insight into discovering knowledge through topic modeling is seeding the knowledge graph with domain concepts to guide the modeling process. We use Wikipedia concepts to provide a basis for performing concept phrase extraction and show how using those phrases improves the quality of the topic models. Researchers can query the resulting knowledge graph to reveal important relations and trends. This work is novel because it uses topics as a bridge to relate documents across corpora over time.

Identifying Topics and Trends in the Study of Common-Pool Resources Using Natural Language Processing

The rapid growth of the literature on the commons poses an immense challenge for the synthesis and advancement of knowledge. While it may have been reasonable for previous generations of scholars to keep up to date with a literature adding thirty to fifty papers each year, there are now hundreds of papers on the commons published each year in addition to those that might be relevant to researchers on the basis of particular sectors, methods, disciplines or theories. This paper exploits recent advances in natural language processing to identify topics and trends in the literature on the commons over the past thirty years using a dynamic topic model. The results highlight the centrality of key themes concerning resources, property rights and local management, alongside growing interest in the topics of conservation and local management. The results also demonstrate the diversity of the field with topics ranging from forests, fisheries and land to urban areas and software. Overall the dynamic topic model appears to provide a useful approach for synthesizing high-level features of the literature.

An evolutionary analysis of new energy and industry policy tools in China based on large-scale policy topic modeling.

This study investigates the evolution of provincial new energy policies and industries of China using a topic modeling approach. To this end, six out of 31 provinces in China are first selected as research samples, central and provincial new energy policies in the period of 2010 to 2019 are collected to establish a text corpus with 23, 674 documents. Then, the policy corpus is fed to two different topic models, one is the Latent Dirichlet Allocation for modeling static policy topics, another is the Dynamic Topic Model for extracting topics over time. Finally, the obtained topics are mapped into policy tools for comparisons. The dynamic policy topics are further analyzed with the panel data from provincial new energy industries. The results show that the provincial new energy policies moved to different tracks after about 2014 due to the regional conditions such as the economy and CO2 emission intensity. Underdeveloped provinces tend to use environment-oriented tools to regulate and control CO2 emissions, while developed regions employ the more balanced policy mix for improving new energy vehicles and other industries. Widespread hysteretic effects are revealed during the correlation analysis of the policy topics and new energy capacity.

The Relationship between Expansion for the e-Vehicle Market and User Communications: The Nissan Leaf Perspective

Purpose - This study identifies a big data analysis of online user comments from social media and describes product sales volume based on media richness theory (MRT) and media synchronicity theory. Design/Methodology/Approach - This study utilized online user comments for electronic vehicles like the Nissan Leaf. The data was used as clustering various sales topics for developing market strategy. Using Word2Vec and Dynamic Topic Modeling, we developed important topics related to Nissan’s market performance with social network analysis. Findings - We found online user responses affected vehicle sales performance with comments on social network sites. Also, dimensions for the communication process on social network services were different from user behaviors. Research Implications - We found online user comments strongly affected automobile industry sales. Thus, firms should consider the power of social network services in the expansion of market volume with conveyance and convergence processes centered around big data in advertisements.

Detecting Topic and Sentiment Trends in Physician Rating Websites: Analysis of Online Reviews Using 3-Wave Datasets.

(1) Background: Physician rating websites (PRWs) are a rich resource of information where individuals learn other people response to various health problems. The current study aims to investigate and analyze the people top concerns and sentiment dynamics expressed in physician online reviews (PORs). (2) Methods: Text data were collected from four U.S.-based PRWs during the three time periods of 2018, 2019 and 2020. Based on the dynamic topic modeling, hot topics related to different aspects of healthcare were identified. Following the hybrid approach of aspect-based sentiment analysis, the social network of prevailing topics was also analyzed whether people expressed positive, neutral or negative sentiments in PORs. (3) Results: The study identified 30 dominant topics across three different stages which lead toward four key findings. First, topics discussed in Stage III were quite different from the earlier two stages due to the COVID-19 outbreak. Second, based on the keyword co-occurrence analysis, the most prevalent keywords in all three stages were related to the treatment, questions asked by patients, communication problem, patients’ feelings toward the hospital environment, disease symptoms, time spend with patients and different issues related to the COVID-19 (i.e., pneumonia, death, spread and cases). Third, topics related to the provider service quality, hospital servicescape and treatment cost were the most dominant topics in Stages I and II, while the quality of online information regarding COVID-19 and government countermeasures were the most dominant topics in Stage III. Fourth, when zooming into the topic-based sentiments analysis, hot topics in Stage I were mostly positive (joy be the dominant emotion), then negative (disgust be the dominant emotion) in Stage II. Furthermore, sentiments in the initial period of Stage III (COVID-19) were negative (anger be the dominant emotion), then transformed into positive (trust be the dominant emotion) later. The findings also revealed that the proposed method outperformed the conventional machine learning models in analyzing topic and sentiment dynamics expressed in PRWs. (4) Conclusions: Methodologically, this research demonstrates the ability and importance of computational techniques for analyzing large corpora of text and complementing conventional social science approaches.

Topic-level sentiment analysis of social media data using deep learning

Due to the inception of Web 2.0 and freedom to facilitate the dissemination of information, sharing views, expressing opinions with regards to current world level events, services, products, etc. social media platforms have been mainly contributing to user-generated content. Such social media data consist of various themes discussed online and are associated with sentiments of the users. To catch up with the speed of streaming data at which it generates on social media platforms, it is crucial to detect the topics being discussed on social media platforms and analyze the sentiments of users towards those topics in an online manner to make timely decisions. Motivated by the same, this paper proposes a deep learning based topic-level sentiment analysis model. The novelty of the proposed approach is that it works at the sentence level to extract the topic using online latent semantic indexing with regularization constraint and then applies topic-level attention mechanism in long short-term memory network to perform sentiment analysis. The proposed model is unique in the sense that it supports scalable and dynamic topic modeling over streaming short text data and performs sentiment analysis at topic-level. For SemEval-2017 Task 4 Subtask B dataset as a case of in-domain topic-level sentiment analysis, average recall of 0.879 has been achieved, whereas, for out-of-domain data, average recall of 0.846, 0.824 and 0.794 has been achieved for newly developed datasets collected under the hashtags #ethereum, #bitcoin and #facebook from Twitter. To assess the performance of the model for scalability, we analyzed the model in terms of average time in milliseconds for creation of feature vectors, throughput in terms of topics detected per second and average response time in seconds to handle the sentiment analysis queries. The experimental results are significant enough to enable large scale topic modeling over streaming data and perform topic-level sentiment analysis.

"Thought I'd Share First" and Other Conspiracy Theory Tweets from the COVID-19 Infodemic: Exploratory Study.

BackgroundThe COVID-19 outbreak has left many people isolated within their homes; these people are turning to social media for news and social connection, which leaves them vulnerable to believing and sharing misinformation. Health-related misinformation threatens adherence to public health messaging, and monitoring its spread on social media is critical to understanding the evolution of ideas that have potentially negative public health impacts.ObjectiveThe aim of this study is to use Twitter data to explore methods to characterize and classify four COVID-19 conspiracy theories and to provide context for each of these conspiracy theories through the first 5 months of the pandemic.MethodsWe began with a corpus of COVID-19 tweets (approximately 120 million) spanning late January to early May 2020. We first filtered tweets using regular expressions (n=1.8 million) and used random forest classification models to identify tweets related to four conspiracy theories. Our classified data sets were then used in downstream sentiment analysis and dynamic topic modeling to characterize the linguistic features of COVID-19 conspiracy theories as they evolve over time.ResultsAnalysis using model-labeled data was beneficial for increasing the proportion of data matching misinformation indicators. Random forest classifier metrics varied across the four conspiracy theories considered (F1 scores between 0.347 and 0.857); this performance increased as the given conspiracy theory was more narrowly defined. We showed that misinformation tweets demonstrate more negative sentiment when compared to nonmisinformation tweets and that theories evolve over time, incorporating details from unrelated conspiracy theories as well as real-world events.ConclusionsAlthough we focus here on health-related misinformation, this combination of approaches is not specific to public health and is valuable for characterizing misinformation in general, which is an important first step in creating targeted messaging to counteract its spread. Initial messaging should aim to preempt generalized misinformation before it becomes widespread, while later messaging will need to target evolving conspiracy theories and the new facets of each as they become incorporated.

Who cares about coal? Analyzing 70 years of German parliamentary debates on coal with dynamic topic modeling

Despite Germany’s Paris Agreement pledge and coal exit legislation, the political debate around carbon-intensive coal remains heated. Coal power and mining have played an important, yet changing role in the history of German politics. In this paper, we analyze the entire parliamentary debate on coal in the German parliament (Bundestag) from its inception in 1949 to 2019. For this purpose we extract the more than 870,000 parliamentary speeches from all protocols in the history of the Bundestag. We identify the 9167 speeches mentioning coal and apply dynamic topic modeling – an unsupervised machine learning technique that reveals the changing thematic structure of large document collections over time – to analyze changes in parliamentary debates on coal over the past 70 years. The trends in topics and their varying internal structure reflect how energy policy was discussed and legitimized over time: Initially, coal was framed as a driver of economic prosperity and guarantee of energy security. In recent years, the debate evolved towards energy transition, coal phase-out and renewable energy expansion. Germany’s smaller and younger parties, the Greens and the Left Party, debate coal more often in the context of the energy transition and climate protection than other parties. Our results reflect trends in other countries and other fields of energy policy. Methodologically, our study illustrates the potential of and need for computational methods to analyze vast corpora of text and to complement traditional social science methods.

Adaptive Framework for Deep Learning Based Dynamic and Temporal Topic Modeling from Big Data

Background: The large amount of data emanated from social media platforms need scalable topic modeling in order to get current trends and themes of events discussed on such platforms. Topic modeling play crucial role in many natural language processing applications like sentiment analysis, recommendation systems, event tracking, summarization, etc. Objective: The aim of the proposed work is to adaptively extract the dynamically evolving topics over streaming data, and infer the current trends and get the notion of trend of topics over time. Because of various world level events, many uncorrelated streaming channels tend to start discussion on similar topics. We aim to find the effect of uncorrelated streaming channels on topic modeling when they tend to start discussion on similar topics. Methods: An adaptive framework for dynamic and temporal topic modeling using deep learning has been put forth in this paper. The framework approximates online latent semantic indexing constrained by regularization on streaming data using adaptive learning method. The framework is designed using deep layers of feedforward neural network. Results: This framework supports dynamic and temporal topic modeling. The proposed approach is scalable to large collection of data. We have performed exploratory data analysis and correspondence analysis on real world Twitter dataset. Results state that our approach works well to extract topic topics associated with a given hashtag. Given the query, the approach is able to extract both implicit and explicit topics associated with the terms mentioned in the query. Conclusion: The proposed approach is a suitable solution for performing topic modeling over Big Data. We are approximating the Latent Semantic Indexing model with regularization using deep learning with differentiable ℓ1 regularization, which makes the model work on streaming data adaptively at real-time. The model also supports the extraction of aspects from sentences based on interrelation of topics and thus, supports aspect modeling in aspect-based sentiment analysis.

Analyzing Research Trends in Blockchain Studies in South Korea Using Dynamic Topic Modeling and Network Analysis

Analyzing Research Trends in Blockchain Studies in South Korea Using Dynamic Topic Modeling and Network Analysis

Dynamic Topic Model for Quantitative Estimation of Conversation Process During Psychotherapy

Dynamic Topic Model for Quantitative Estimation of Conversation Process During Psychotherapy

Supporting secondary research in early drug discovery process through a Natural Language Processing based system

Abstract Last decades were characterised by a constant decline in the productivity of research and development activities of pharmaceutical companies. This is due to the fact that the drug discovery process contains an intrinsic risk that should be managed efficiently. Within this process, the early phase projects could be streamlined by doing more secondary research. These activities would involve the integration of chemical and biological knowledge from scientific literature in order to extract an overview and the evolution of a certain research area. This would then help refine the research and development operations. Considering the vast amount of pharmaceutical studies publications, it is not easy to identify the important information. For this task, a series of projects leveraged the advantages of the open pharmacological space through state-of-the-art technologies. The most popular are Knowledge Graphs methods. Although extremely useful, this technology requires increased investments of time and human resources. An alternative would be to develop a system that uses Natural Language Processing blocks. Still, there is no defined framework and reusable code template for the use-case of compounds development. In this study, it is presented the design and development of a system that uses Dynamic Topic Modelling and Named Entity Recognition modules in order to extract meaningful information from a large volume of unstructured texts. Moreover, the dynamic character of the topic modelling technique allows to analyse the evolution of different subject areas over time. In order to validate the system, a collection of articles from the Pharmaceutical Research Journal was used. Our results show that the system is able to identify the main research areas in the last 20 years, namely crystalline and amorphous systems, insulin resistance, paracellular permeability. Additionally, the evolution of the subjects is a highly valuable resource and should be used to get an in-depth understanding about the shifts that happened in a specific domain. However, a limitation of this system is that it cannot detect association between two concepts or entities if they are not involved in the same document.

Supporting secondary research in early drug discovery process through a Natural Language Processing based system

Abstract Last decades were characterised by a constant decline in the productivity of research and development activities of pharmaceutical companies. This is due to the fact that the drug discovery process contains an intrinsic risk that should be managed efficiently. Within this process, the early phase projects could be streamlined by doing more secondary research. These activities would involve the integration of chemical and biological knowledge from scientific literature in order to extract an overview and the evolution of a certain research area. This would then help refine the research and development operations. Considering the vast amount of pharmaceutical studies publications, it is not easy to identify the important information. For this task, a series of projects leveraged the advantages of the open pharmacological space through state-of-the-art technologies. The most popular are Knowledge Graphs methods. Although extremely useful, this technology requires increased investments of time and human resources. An alternative would be to develop a system that uses Natural Language Processing blocks. Still, there is no defined framework and reusable code template for the use-case of compounds development. In this study, it is presented the design and development of a system that uses Dynamic Topic Modelling and Named Entity Recognition modules in order to extract meaningful information from a large volume of unstructured texts. Moreover, the dynamic character of the topic modelling technique allows to analyse the evolution of different subject areas over time. In order to validate the system, a collection of articles from the Pharmaceutical Research Journal was used. Our results show that the system is able to identify the main research areas in the last 20 years, namely crystalline and amorphous systems, insulin resistance, paracellular permeability. Additionally, the evolution of the subjects is a highly valuable resource and should be used to get an in-depth understanding about the shifts that happened in a specific domain. However, a limitation of this system is that it cannot detect association between two concepts or entities if they are not involved in the same document.

The Prediction of Sales Volume and WoM Effect Based on Topic Extraction from Social Media

Purpose - This study attempts to predict the automobile sales volume and WoM Effect of electronic cars such as the Tesla Model S. Design/Methodology/Approach - This study extracted the data in a new industry, specifically, an electronic car such as the Tesla Model S based on the literature review of the existing media theory: the Media Richness Theory (MRT) and the Media Synchronicity Theory (MST). The unstructured data was used as text data for classifying important topics following the flow of time. Using Latent Dirichlet Allocation (LDA) and Dynamic Topic Modeling (DTM), we deducted the significant topic related to the company performance such as sales volume. Findings - We find out that the text mining approach allowed us to identify the derived indicators for an automobile manufacturer’s market success with centrality measures. Also, this research found evidence of conveyance and convergence for the communication process on the web relative to users’ web comments. Research Implications - Although previous studies used the economic approach, this study established that social media could capitalize on using conveyance and convergence of the communication process (e.g., conveyance vs. convergence) based on MST through theoretical consideration.