Topic Tracking Research Articles

A Social Group Chatbot System by Multiple Topics Tracking and Atkinson‐Shiffrin Memory Model Using AI Agents Collaboration

ABSTRACTThe widespread use of Internet has accelerated the explosive growth of data, which in turn leads to information overload and information confusion. This makes it difficult for us to communicate effectively in social groups, thereby intensifying the demands for emotional companionship. Therefore, we propose a novel social group chatting framework based on Large Language Model (LLM) powered multiple autonomous agents collaboration in this article. Specifically, BERTopic is used to extract topics from history chatting content for each social group everyday, and then multiple topics tracking is realised through multi‐level association by adaptive time sliding‐window mechanism and optimal matching. Furthermore, we use topic tracking architecture and prompts to design and implement an AI Chatbot system with different characters that can conduct natural language conversations with users in online social group. LLM, as the controller and coordinator of the whole AI Chatbot for sub‐tasks, allows different AI Agents to autonomously decide whether to participate in current topic, how to generate response, and whether to propose a new topic. Each AI Agent has their own multi‐store memory system based on the Atkinson‐Shiffrin model. Finally, we construct a verification environment based on online game that is consistent with real society. Subjective and objective evaluation methods were deployed to perform qualitative and quantitative analyses to demonstrate the performance of our AI Chatbot system.

Classification and topic tracking of college students’ cybersecurity education based on the internet

Campus security issues have always been a hot topic of concern from all walks of life. To analyze the changes in network public opinion of college students’ campus security incidents, this study proposes to classify campus security incidents and track related topics. The classification algorithm and topic tracking algorithm for campus security incidents based on Internet technology are studied, and a prototype system of campus security incidents based on Internet is designed for analysis. The research results show that the Internet-based campus security incident prototype system can classify campus security incidents more accurately, and can also track security incidents on related topics, which provide support for student security management to a certain extent. This technology will be applied to the tracking and classification of college safety accidents, and provide certain reference for ensuring college students’ campus safety.

A Review on the Analysis of Public Opinion Regarding Natural Disasters

Natural disasters pose as one of the paramount challenges faced globally, exerting immense influence upon society, economy, and the environment. With the proliferation of social media, online public opinion has emerged as a pivotal source of information for studying natural disasters. Analysis of public opinion surrounding natural disasters holds significant implications in offering decision-making support and facilitating risk management. This review aims to recapitulate and analyze the recent advancements in the study of public opinion regarding natural disasters. Initially, It provides a summary of the relevant methods and techniques by delving into the modules of topic detection and topic tracking. Subsequently, it explores the applications of public opinion analysis, encompassing disaster monitoring and early warning, post-disaster assessment and emergency response, as well as the analysis of public opinion evolution. Finally, It elucidates the challenges and prospects in the field of public opinion analysis, encompassing aspects such as multimodal data fusion, enhancement of algorithmic efficacy, as well as cross-lingual and cross-cultural public opinion analysis. In conclusion, public opinion analysis can assist governments and relief agencies in gaining a deeper understanding of the public's cognition, emotions, and needs in relation to disasters, thereby enabling the formulation of more effective response measures.

Microblog topic detection using evolutionary clustering and social network information

SummaryThe messages in microblogs, like Twitter, present a wealth of information around the hot topics and public opinions. Topic detection and tracking enables a better understanding of the incoming information from social networks. Many of the studies performed in this field consider a set of predefined number of topics (clusters) which remain constant during the detection process. Such approaches are not efficient when data is dynamic and cumulative. In addition, nonparametric evolutionary topic models do not have an accurate performance on short texts due to the data deficiency. In this article, we propose an evolutionary clustering model based on the Bayesian nonparametric Dirichlet process mixture model, which automatically determines the number of clusters. To improve the algorithm performance when dealing with short social media texts, temporal information of tweets and other social network parameters are used in a weighted combination to augment the text similarity in detecting topics. Model evaluation is performed on the data gathered from the Twitter during a 2.5‐month period. Results reveal that the social network information can promote the topic detection performance. The proposed model outperforms previous study in terms of topic coherence and clustering performance, leading to a better clustering on short texts.

An Unsupervised Graph-Based Approach for Detecting Relevant Topics: A Case Study on the Italian Twitter Cohort during the Russia–Ukraine Conflict

On 24 February 2022, the invasion of Ukraine by Russian troops began, starting a dramatic conflict. As in all modern conflicts, the battlefield is both real and virtual. Social networks have had peaks in use and many scholars have seen a strong risk of disinformation. In this study, through an unsupervised topic tracking system implemented with Natural Language Processing and graph-based techniques framed within a biological metaphor, the Italian social context is analyzed, in particular, by processing data from Twitter (texts and metadata) captured during the first month of the war. The system, improved if compared to previous versions, has proved to be effective in highlighting the emerging topics, all the main events and any links between them.

Research on Efficient Work Innovation Effect Evaluation Based on Real-Time Online Public Opinion Data Classification and Detection Algorithm

Various heterogeneous network resources are collected through web crawlers, and after preprocessing such as data cleaning, they are written into the data center. And implement data mining technologies such as classification, clustering, association rules and anomaly detection. If an enterprise wants to maintain a good development ability and maintain a correct development trend, it must pay attention to and do a good job in ideological and political work, and take ideological and political work as the guiding light of its own operation and development. Improved the stability and accuracy of clustering results. The news topics are clustered to form topics, and the topic tracking technology is used to track specific news topics to achieve personalized services.

Ad-Hoc Monitoring of COVID-19 Global Research Trends for Well-Informed Policy Making

The COVID-19 pandemic has affected millions of people worldwide with severe health, economic, social, and political implications. Healthcare Policy Makers (HPMs) and medical experts are at the core of responding to this continuously evolving pandemic situation and are working hard to contain the spread and severity of this relatively unknown virus. Biomedical researchers are continually discovering new information about this virus and communicating the findings through scientific articles. As such, it is crucial for HPMs and funding agencies to monitor the COVID-19 research trend globally on a regular basis. However, given the influx of biomedical research articles, monitoring COVID-19 research trends has become more challenging than ever, especially when HPMs want on-demand guided search techniques with a set of topics of interest in mind. Unfortunately, existing topic trend modeling techniques are unable to serve this purpose as (1) traditional topic models are unsupervised, and (2) HPMs in different regions may have different topics of interest that they want to track. To address this problem, we introduce a novel computational task in this article called Ad-Hoc Topic Tracking , which is essentially a combination of zero-shot topic categorization and the spatio-temporal analysis task. We then propose multiple zero-shot classification methods to solve this task by building on state-of-the-art language understanding techniques. Next, we picked the best-performing method based on its accuracy on a separate validation dataset and then applied it to a corpus of recent biomedical research articles to track COVID-19 research endeavors across the globe using a spatio-temporal analysis. A demo website has also been developed for HPMs to create custom spatio-temporal visualizations of COVID-19 research trends. The research outcomes demonstrate that the proposed zero-shot classification methods can potentially facilitate further research on this important subject matter. At the same time, the spatio-temporal visualization tool will greatly assist HPMs and funding agencies in making well-informed policy decisions for advancing scientific research efforts.

Identifying chronological and coherent information threads using 5W1H questions and temporal relationships

Due to the massive volume of articles produced online every day, it is challenging for online platforms (e.g., news agencies) to present the information about an event, activity or discussion to their users in an easily digestible format. Therefore, there is a need for automatic methods to extract related and time-ordered information about events (i.e., information threads) from large unstructured collections of documents. In this work, we propose a novel unsupervised hierarchical agglomerative clustering (HAC) based information threading approach to generate chronological and coherent threads of information in a collection. Unlike, the well-known tasks of topic detection and tracking or event threading that focus on grouping information by important keywords and/or entities, our proposed approach identifies threads based on temporal relations and diverse information about an event, i.e., who did what, why, where, when and how (aka the 5W1H questions). In particular, our proposed approach, deploys a tailored similarity function for HAC by leveraging extracted answers to 5W1H questions along with time decay between documents. We evaluate our proposed HAC 5W1H information threading approach on two large expert-annotated collections of news articles, i.e., NewSHead and Multi-News (over 112k and 32k articles, respectively). Our experiments show that HAC 5W1H markedly improves the number of, and quality of, threads that are generated compared to existing state-of-the-art approaches from the literature, e.g., 100.98% more threads and +213.39% improvement in Normalised Mutual Information compared to the best evaluated baseline on the larger NewSHead collection. We also conducted a user study that shows that our proposed HAC 5W1H information threading approach is significantly (p<0.05) preferred by users in terms of coherence, diversity and chronological correctness compared to the existing state-of-the-art approaches.

Keyword Extraction: A Modern Perspective

The goal of keyword extraction is to extract from a text, words, or phrases indicative of what it is talking about. In this work, we look at keyword extraction from a number of different perspectives: Statistics, Automatic Term Indexing, Information Retrieval (IR), Natural Language Processing (NLP), and the emerging Neural paradigm. The 1990s have seen some early attempts to tackle the issue primarily based on text statistics [13, 17]. Meanwhile, in IR, efforts were largely led by DARPA’s Topic Detection and Tracking (TDT) project [2]. In this contribution, we discuss how past innovations paved a way for more recent developments, such as LDA, PageRank, and Neural Networks. We walk through the history of keyword extraction over the last 50 years, noting differences and similarities among methods that emerged during the time. We conduct a large meta-analysis of the past literature using datasets from news media, science, and medicine to business and bureaucracy, to draw a general picture of what a successful approach would look like.

Document clustering analysis with aid of adaptive Jaro Winkler with Jellyfish search clustering algorithm

In this research, document clustering is analyzed with the help of Adaptive Jaro Winkler with Jellyfish Search Clustering (AJWJSC) algorithm and Chimp Optimization Algorithm (COA). The major motive of the research is to compute the relevant topics with the easiest way in addition reduce the complexity of the domain analysis. The document retrieval process is analysis for the recent topic detection and identification. In the research, four stages are considered to analyze the documents, named as pre-processing, feature extraction, feature knowledge establishment, in addition document clustering phase. Initially, the documents can be pre-processed with the consideration on tokenization, stop-word removal and stemming methods. After that, the Relative Document-Term Frequency Difference (RD-TFD) technique can be utilized to extract the features. Based on the extracted feature set, the essential features are nominated with the help of Chimp Optimization Algorithm (COA). Afterwards, the document clustering process is computed using the Adaptive Jaro Winkler with Jellyfish Search Clustering (AJWJSC) algorithm. The novelty of the work is to document clustering from the documents for utilized in different applications. The proposed method is designed in the MATLAB platform and analyzed with the three datasets named as Reuter database, 20 Newsgroups and Topic detection and tracking (TDT2) database respectively. While analyzing the statistical measures of the research, various parameters is determined like precision, Recall, F-measure, accuracy, and the efficiency. Here, the clustered documents are validated based on their similarity and this is used for the document retrieval purpose. The projected technique can be contrasted with the k-means Clustering, Krill herd (KH) algorithm, and Moth Flame Optimization (MFO) Algorithm respectively.

DAC: Descendant-aware clustering algorithm for network-based topic emergence prediction

• Authors are more topically similar when they are from multigraph clusters. • Changes in the definition of topics lead to more human-readable topics. • Topics from only meta-data of the publication records show higher topical coherence. • Use of multiple bibliographic networks is beneficial to author-based topic modeling. • Author-based topic modeling allows effective merge/split topic evolution tracking. Topic emergence detection aids in pinpointing prominent topics within a given domain, providing practical insights into all interested parties on where to focus the limited resources. This paper employs the network-based topic evolution approach to overcome limitations in text-based topic evolution, providing prospective topic emergence prediction capabilities by representing emergent topics by their ancestors. A descendant-aware clustering algorithm is proposed to generate non-exhaustive and overlapping clusters, utilizing the pace of collaborations and structural similarities between topics with iterative edge removal and addition processes. Over 100 datasets specific to a research topic were extracted from the Microsoft Academic Graph dataset for the experiments, where the proposed algorithm consistently outperformed existing clustering algorithms in generating clusters with a higher likelihood of being ancestors to an emergent topic up to three years in the future. Regression-based cluster filtering using five structural cluster features and topic cluster qualities showed that the prediction performance can be enhanced by automatically classifying undesirable clusters from previously known data. The results showed that the proposed algorithm can enhance topic emergence predictions on a wide range of research domains regardless of their maturities, popularities, and magnitudes without having access to the data in the predicted year, paving a road to prospective predictions on emergent topics.

ECANP: A Topic Influence Evaluation Model for Hot Topics.

Social network is an important product of industrial society. In recent years, the research related to hot topics has focused on topic detection, topic trend prediction, and topic tracking. However, the important role of topic influence evaluation in hot topic research has not received enough attention, which leads the problem of inaccurate influence calculation. In order to solve the above problems, this paper proposes a novel model to evaluate the real-time relative influence of topics in social network. The proposed model can quantify the influence of topics, and some influential factors which determine topic hotness will be analyzed and identified. In this model, five impact indicators are defined, namely user engagement, topic coverage, topic activity, topic persistence, and topic novelty to consider the topic characteristics more finely. Moreover, the proposed model not only consider traditional simple factors of like, forward and comments, but also pay attention to the relative influence and time attenuation characteristics of the topics. Further, the experimental results show that our method could quickly aggregate the influence factors of hot topics and accurately provide the influence indicator of topics.

A Performance Comparison of Unsupervised Techniques for Event Detection from Oscar Tweets.

People's lives are influenced by social media. It is an essential source for sharing news, awareness, detecting events, people's interests, etc. Social media covers a wide range of topics and events to be discussed. Extensive work has been published to capture the interesting events and insights from datasets. Many techniques are presented to detect events from social media networks like Twitter. In text mining, most of the work is done on a specific dataset, and there is the need to present some new datasets to analyse the performance and generic nature of Topic Detection and Tracking methods. Therefore, this paper publishes a dataset of real-life event, the Oscars 2018, gathered from Twitter and makes a comparison of soft frequent pattern mining (SFPM), singular value decomposition and k-means (K-SVD), feature-pivot (Feat-p), document-pivot (Doc-p), and latent Dirichlet allocation (LDA). The dataset contains 2,160,738 tweets collected using some seed words. Only English tweets are considered. All of the methods applied in this paper are unsupervised. This area needs to be explored on different datasets. The Oscars 2018 is evaluated using keyword precision (K-Prec), keyword recall (K-Rec), and topic recall (T-Rec) for detecting events of greater interest. The highest K-Prec, K-Rec, and T-Rec were achieved by SFPM, but they started to decrease as the number of clusters increased. The lowest performance was achieved by Feat-p in terms of all three metrics. Experiments on the Oscars 2018 dataset demonstrated that all the methods are generic in nature and produce meaningful clusters.

A Framework for Topic Evolution and Tracking Their Sentiments With Time

With the ongoing covid-19 pandemic, people rely on online communication to remain connected as a precautionary measure to maintain social distancing. When we have no one on our side to listen and console us in state of fear and dilemma, we try to find comfort in anonymity of social media. Tracking real-time changes in sentiments are quite difficult as it could not correlate well with human understanding and emotions, which changes with time and many other factors. Collecting sentiments from users on search results, news articles, paintings, photographs are nowadays common. This is a more robust and effective method as traditional ways do not rely on a lot of retrospectives. In this paper, we will be analyzing the data collected from Twitter on Covid-19 and see topic modelling can be meant to detect sentiment analysis. The challenge is here we need to see results over time, and changes detect in topics and sentiments. We analyze our method over covid-19 data and farmer’s protest. Results from this experiment using the proposed methodology are promising and giving valuable insights.

Community-Level Social Topic Tracking of Urban Emergency: A Case Study of COVID-19

During the outbreak of COVID-19 in Wuhan, the public used Weibo to acquire information, express emotions, and seek help at a historic and unprecedented scale. This situational information is valuable for authorities to initiate emergency policies and protect people’s health. Although event detection and situation awareness during COVID-19 using social media data have been investigated in recent studies, the spatiotemporal evolution patterns of social topics during COVID-19 at the community level are still limited. To address this limitation, this study developed a methodology to explore the dynamics of social topics under the influence of COVID-19 at a community level in Wuhan using geotagged Weibo posts. The disease-related topics exhibited the spatial variations in different urban communities and reflected the dynamics of public attention. Moreover, we used the reported COVID-19 case data in Wuhan and geotagged Weibo posts in Beijing to validate the effectiveness of the proposed methodology. These findings help to track social topic evolution at the community level and gain a better understanding of the spatial variations of public attention during urban emergencies.

Analysis of the dynamics and influence of the research work of Prof. Liu Zeyuan in China featuring a new hybrid approach combining community detection with topic tracking

The influence of authors is mostly based on their capacity to form specific self-contained and/or active research communities or topics while also inspiring fruitful spin-off research derived from those communities or topics. Accurately estimating author influence and its indirect effect in inspiring external research creativity must thus be based on very precise community roles as well as topic-role analysis on citing material. In this paper we thus implement, compare, and combine two different kinds of knowledge mapping approaches used to track authors’ influence over time—a recent topic-based mapping approach and an original hybrid community-based mapping approach. The experimental context of our study is the analysis of the dynamics and of the influence of the research of Prof. Liu Zeyuan, the most important contributor in the field of Science of Science in China. This analysis focuses on papers citing Professor Liu Zeyuan's research and highlights the full extent of his creative thinking.

Evaluating Cultural Impact in Discursive Space through Digital Footprints

The research presented in this paper describes an evaluation of the impact of spatial interventions in public spaces, measured by social media data. This contribution aims at observing the way a spatial intervention in an urban location can affect what people talk about on social media. The test site for our research is Domplatz in the center of Hamburg, Germany. In recent years, several actions have taken place there, intending to attract social activity and spotlight the square as a landmark of cultural discourse in the city of Hamburg. To evaluate the impact of this strategy, textual data from the social networks Twitter and Instagram (i.e., tweets and image captions) are collected and analyzed using Natural Language Processing intelligence. These analyses identify and track the cultural topic or “people talking about culture” in the city of Hamburg. We observe the evolution of the cultural topic, and its potential correspondence in levels of activity, with certain intervention actions carried out in Domplatz. Two analytic methods of topic clustering and tracking are tested. The results show a successful topic identification and tracking with both methods, the second one being more accurate. This means that it is possible to isolate and observe the evolution of the city’s cultural discourse using NLP. However, it is shown that the effects of spatial interventions in our small test square have a limited local scale, rather than a city-wide relevance.

A Similarity Algorithm Based on Hash Feature and Random Mapping

Similar text algorithm is the main method for news topic extraction and online public opinion hot spot tracking. The traditional text similarity method has the problems of high data dimension and high computational complexity. This paper proposes a method of news text clustering based on hash feature and random mapping. The method based on hash feature reduces the dimensionality of the text. At the same time, the article clustering preprocessing based on random mapping can speed up retrieval. Experiments show that the algorithm proposed in this paper performs well on related data sets.

An Automatic Participant Detection Framework for Event Tracking on Twitter

Topic Detection and Tracking (TDT) on Twitter emulates human identifying developments in events from a stream of tweets, but while event participants are important for humans to understand what happens during events, machines have no knowledge of them. Our evaluation on football matches and basketball games shows that identifying event participants from tweets is a difficult problem exacerbated by Twitter’s noise and bias. As a result, traditional Named Entity Recognition (NER) approaches struggle to identify participants from the pre-event Twitter stream. To overcome these challenges, we describe Automatic Participant Detection (APD) to detect an event’s participants before the event starts and improve the machine understanding of events. We propose a six-step framework to identify participants and present our implementation, which combines information from Twitter’s pre-event stream and Wikipedia. In spite of the difficulties associated with Twitter and NER in the challenging context of events, our approach manages to restrict noise and consistently detects the majority of the participants. By empowering machines with some of the knowledge that humans have about events, APD lays the foundation not just for improved TDT systems, but also for a future where machines can model and mine events for themselves.

Is it feasible to detect FLOSS version release events from textual messages? A case study on Stack Overflow.

Topic Detection and Tracking (TDT) is a very active research question within the area of text mining, generally applied to news feeds and Twitter datasets, where topics and events are detected. The notion of "event" is broad, but typically it applies to occurrences that can be detected from a single post or a message. Little attention has been drawn to what we call "micro-events", which, due to their nature, cannot be detected from a single piece of textual information. The study investigates the feasibility of micro-event detection on textual data using a sample of messages from the Stack Overflow Q&A platform and Free/Libre Open Source Software (FLOSS) version releases from Libraries.io dataset. We build pipelines for detection of micro-events using three different estimators whose parameters are optimized using a grid search approach. We consider two feature spaces: LDA topic modeling with sentiment analysis, and hSBM topics with sentiment analysis. The feature spaces are optimized using the recursive feature elimination with cross validation (RFECV) strategy. In our experiments we investigate whether there is a characteristic change in the topics distribution or sentiment features before or after micro-events take place and we thoroughly evaluate the capacity of each variant of our analysis pipeline to detect micro-events. Additionally, we perform a detailed statistical analysis of the models, including influential cases, variance inflation factors, validation of the linearity assumption, pseudo R2 measures and no-information rate. Finally, in order to study limits of micro-event detection, we design a method for generating micro-event synthetic datasets with similar properties to the real-world data, and use them to identify the micro-event detectability threshold for each of the evaluated classifiers.