Social Data Streams Research Articles

Retraction Note: CanarDeep: a hybrid deep neural model with mixed fusion for rumour detection in social data streams

Retraction Note: CanarDeep: a hybrid deep neural model with mixed fusion for rumour detection in social data streams

H-Louvain: Hierarchical Louvain-based community detection in social media data streams

H-Louvain: Hierarchical Louvain-based community detection in social media data streams

Unsupervised social event detection via hybrid graph contrastive learning and reinforced incremental clustering

Detecting events from social media data streams is gradually attracting researchers. The innate challenge for detecting events is to extract discriminative information from social media data thereby assigning the data into different events. Due to the excessive diversity and high updating frequency of social data, using supervised approaches to detect events from social messages is hardly achieved. To this end, recent works explore learning discriminative information from social messages by leveraging graph contrastive learning (GCL) and embedding clustering in an unsupervised manner. However, two intrinsic issues exist in benchmark methods: conventional GCL can only roughly explore partial attributes, thereby insufficiently learning the discriminative information of social messages; for benchmark methods, the learned embeddings are clustered in the latent space by taking advantage of certain specific prior knowledge, which conflicts with the principle of unsupervised learning paradigm. In this paper, we propose a novel unsupervised social media event detection method via hybrid graph contrastive learning and reinforced incremental clustering (HCRC), which uses hybrid graph contrastive learning to comprehensively learn semantic and structural discriminative information from social messages and reinforced incremental clustering to perform efficient clustering in a solidly unsupervised manner. We conduct comprehensive experiments to evaluate HCRC on the Twitter and Maven datasets. The experimental results demonstrate that our approach yields consistent significant performance boosts. In traditional incremental setting, semi-supervised incremental setting and solidly unsupervised setting, the model performance has achieved maximum improvements of 53%, 45%, and 37%, respectively.

Real-Time Event Detection in Social Media Streams using Stream Data Mining

Abstract: The need for real-time event detection has grown rapidly in the modern world of instantaneous information transmission through social media. In order to implement real-time event detection inside the dynamic environment of social media streams, this research article provides a ground-breaking framework that harnesses the power of stream data mining techniques. The combination of three different stream data mining algorithms—Sliding Window Analysis, Burst Detection using K-Means, and Agglomerative Hierarchical Clustering—allows us to tackle this problem. Together, these algorithms make it possible to extract important patterns, shedding light on how events arise inside social media streams. Utilizing cutting-edge stream data mining techniques, this study introduces a novel framework for real-time event detection within social media streams. The quick identification and monitoring of real-world events take on critical importance in the modern environment of rapid information dissemination through social media channels. The dynamic and high-velocity characteristics of social media data streams present difficulties for conventional event detection methodologies.

Application of AI in crisis management

Purpose: Artificial intelligence (AI) can be applied in various areas of crisis management. It can provide numerous benefits concerning speed, effectiveness and precision of actions. AI accelerates the process of massive amounts of data processing, such as sensor data, satellite images, social data streams and much more. This enables to make quick analysis of a situation and make quick decisions in order to monitor various sources of information in real-time and detect early signals of a crisis. Advanced algorithms can identify patterns and anomalies enabling a quick response. AI can be used to model various crisis scenarios based on available historical and current data. This way it is possible to predict the probable effects of a given crisis and prepare for various situations. AI can analyze social media data to monitor community sentiments, identify potential threats and manage crisis communication. Automatic sentiment analysis systems can help understand society's response to a crisis. Advanced AI systems can be used to predict human behavior in various crisis scenarios. This allows to understand better how people may react and what their needs are in a given time. AI can help detect and combat cyber attacks, which constitute a serious threat, especially when it is necessary to maintain critical infrastructures. Integrating AI into crisis management requires caution because decisions made by AI systems should be consistent with ethical standards and they must consider the human supervision and assessment of the crisis situation. Design/methodology/approach: The paper analyzes the AI methods applied during crises. The results of interviews conducted with students, teachers and authors' personal experiences are provided. Findings: AI is a new way of solving problems, especially during crises. The new technologies determine the way and the speed of problems solving. Originality/value: Originality based on the interviews and post-crisis (COVID) points of view are the value. Keywords crises, AI, Chat GPT. Category of the paper: Research paper, Viewpoint.

A New Big Data Architecture for Analysis: The Challenges on Social Media

The streams of social media big data are now becoming an important issue. But the analytics method and tools for this data may not be able to find the useful information from this massive amount of data. The question then becomes: how do we create a high-performance platform and a method to efficiently analyse social networks’ big data; how to develop a suitable mining algorithm for finding useful information from social media big data. In this work, we propose a new hierarchical big data analysis for understanding human interaction, and we present a new method to measure the useful tweets of Twitter users based on the three factors of tweet texts. Finally, we use this test implementation score, in order to detect useful and classification tweets by interested degree.

Metaheuristic enabled hot event detection and product recommendation in social media data streams

Metaheuristic enabled hot event detection and product recommendation in social media data streams

How Twitter data sampling biases U.S. voter behavior characterizations.

Online social media are key platforms for the public to discuss political issues. As a result, researchers have used data from these platforms to analyze public opinions and forecast election results. The literature has shown that due to inauthentic actors such as malicious social bots and trolls, not every message is a genuine expression from a legitimate user. However, the prevalence of inauthentic activities in social data streams is still unclear, making it difficult to gauge biases of analyses based on such data. In this article, we aim to close this gap using Twitter data from the 2018 U.S. midterm elections. We propose an efficient and low-cost method to identify voters on Twitter and systematically compare their behaviors with different random samples of accounts. We find that some accounts flood the public data stream with political content, drowning the voice of the majority of voters. As a result, these hyperactive accounts are over-represented in volume samples. Hyperactive accounts are more likely to exhibit various suspicious behaviors and to share low-credibility information compared to likely voters. Our work provides insights into biased voter characterizations when using social media data to analyze political issues.

Deep-Eware: spatio-temporal social event detection using a hybrid learning model

Event detection from social media aims at extracting specific or generic unusual happenings, such as, family reunions, earthquakes, and disease outbreaks, among others. This paper introduces a new perspective for the hybrid extraction and clustering of social events from big social data streams. We rely on a hybrid learning model, where supervised deep learning is used for feature extraction and topic classification, whereas unsupervised spatial clustering is employed to determine the event whereabouts. We present ‘Deep-Eware’, a scalable and efficient event-aware big data platform that integrates data stream and geospatial processing tools for the hybrid extraction and dissemination of spatio-temporal events. We introduce a pure incremental approach for event discovery, by developing unsupervised machine learning and NLP algorithms and by computing events’ lifetime and spatial spanning. The system integrates a semantic keyword generation tool using KeyBERT for dataset preparation. Event classification is performed using CNN and bidirectional LSTM, while hierarchical density-based spatial clustering was used for location-inference of events. We conduct experiments over Twitter datasets to measure the effectiveness and efficiency of our system. The results demonstrate that this hybrid approach for spatio-temporal event extraction has a major advantage for real-time spatio-temporal event detection and tracking from social media. This leads to the development of unparalleled smart city applications, such as event-enriched trip planning, epidemic disease evolution, and proactive emergency management services.

E-ware: a big data system for the incremental discovery of spatio-temporal events from microblogs.

Event detection from social media aims at extracting specific or generic unusual happenings, such as, family reunions, earthquakes, and disease outbreaks, among others. This paper introduces a new perspective for the incremental extraction and clustering of social events from big social data streams. We present ‘E-ware’, a scalable and efficient big data platform that integrates data stream and geospatial processing tools for the incremental extraction and dissemination of spatio-temporal events. We introduce a pure incremental approach for event discovery, by developing unsupervised machine learning and NLP algorithms and by computing events’ lifetime and spatial spanning. Our incremental clustering technique employs temporal sliding windows, in order to update the discovered topic clusters with the upcoming social streams (i.e., tweets). The system integrates an efficient spatio-temporal index for fast retrieval and updates of evolving event clusters. We conduct experiments over Twitter datasets to measure the effectiveness and efficiency of our system. The results demonstrate that E-ware has a major advantage for real-time incremental detection and tracking of events, both spatially and temporally. This leads to the development of unparalleled smart city applications, such as event-enriched trip planning, epidemic disease evolution, and proactive emergency management services.

Modified valence aware dictionary for sentiment reasoning classifier for detection and classification of Covid‐19 related rumors from social media data streams

SummaryThe perpetual increase in social media data due to social distancing policy in practice and the low cost of communication has acquired an interest in rumor detection in social media. During the togetherness of the globalized world in the pandemic situation of Covid‐19, social media applications drive excellent value to the analysts and journalists working in different domains. However, the unmoderated nature of social media users often leads to the spread of rumors due to a variety of opinions on government decisions, which becomes notoriously hard to detect. Also, the users may deny rumors as soon as they are debunked. This research extracts diffused information such as word cloud, hashtags, re‐tweets and so forth and uses it as features. These features are applied in the sliding windows of data streams to detect tweets that may be rumors by validating using credible news sources. To address the challenge of veracity, tweets are classified into rumors and nonrumors. A modified unsupervised VADER sentiment classifier is proposed to further classify rumors tweets into amalgamated, unauthoritative, or exaggerated. It is revealed from the results that the proposed classifier is quite efficient in finding and classifying rumorous tweets that arise in critical situations.

A framework for social media information flow analytics in cyberspace and physical space

AbstractCommunication in online social media naturally forms information flow from senders to receivers. With the proliferation of location‐aware social media services such as Twitter, information flow shaped on these platforms is becoming more deeply and extensively embedded in the physical world. Therefore, growing interest has been raised in discovering the spatiotemporal patterns of information flow in location‐aware social media, aiming for a holistic understanding of the social dynamics in the nested cyber and physical world. This article addresses this interest by proposing a framework for information flow analytics based on a spatiotemporal clustering method designed for large data streams with location information. The framework was implemented as an open‐source tool, and its performance was examined by two case studies using Twitter data streams. It is suggested that this framework has well addressed the new requirements and challenges arising from social media data streams, effectively visualized information flow on maps, and demonstrated its feasibility of supporting downstream spatiotemporal analysis and applications.

CanarDeep: a hybrid deep neural model with mixed fusion for rumour detection in social data streams.

The unrelenting trend of doctored narratives, content spamming, fake news and rumour dissemination on social media can lead to grave consequences that range from online intimidating and trolling to lynching and riots in real- life. It has therefore become vital to use computational techniques that can detect rumours, do fact-checking and inhibit its amplification. In this paper, we put forward a model for rumour detection in streaming data on social platforms. The proposed CanarDeep model is a hybrid deep neural model that combines the predictions of a hierarchical attention network (HAN) and a multi-layer perceptron (MLP) learned using context-based (text + meta-features) and user-based features, respectively. The concatenated context feature vector is generated using feature-level fusion strategy to train HAN. Eventually, a decision-level late fusion strategy using logical OR combines the individual classifier prediction and outputs the final label as rumour or non-rumour. The results demonstrate improved performance to the existing state-of-the-art approach on the benchmark PHEME dataset with a 4.45% gain in F1-score. The model can facilitate well-time intervention and curtail the risk of widespread rumours in streaming social media by raising an alert to the moderators.

Edge intelligence‐enabled dynamic overlapping community discovery and evolution prediction in social media data streams

AbstractEdge intelligence (EI) is recognized by academia and industry as one of the key emerging technologies for future cyber‐physical‐social systems (CPSS), which provides the ability to analyze data at edge rather than sending it to the cloud for analysis, and will be a key enabler to realize a world of a trillion hyper‐connected smart sensing devices. As a part of future CPSS, online social networks are large‐scale complex networks that consist of a large number of network nodes and links. The dynamic discovery of communities, especially overlapping communities, is important to understand the evolution of online social networks. However, traditional community discovery algorithms cannot effectively discover overlapping communities in social networks. In order to address this challenge, an edge intelligence‐enabled dynamic overlapping community discovery and evolution prediction model (EIDEP) is proposed in this article. This model encompasses a label propagation algorithm based extension (LPAE) algorithm, which is able to efficiently discover the user community structures in online social networks. Based on the LPAE community discovery algorithm, a user interest behavior based evolution prediction (UIBEP) algorithm is incorporated in our EIDEP model in order to realize a fast yet accurate community evolution for online social networks, by considering the interest similarity of unlinked nodes in a given community. The performance of our proposed LPAE and UIBEP models is validated and evaluated against notable state‐of‐the‐art community discovery algorithms, through extensive experiments conducted based on a Twitter dataset.

SHEDR: An End-to-End Deep Neural Event Detection and Recommendation Framework for Hyperlocal News Using Social Media

Residents often rely on newspapers and television to gather hyperlocal news for community awareness and engagement. More recently, social media have emerged as an increasingly important source of hyperlocal news. Thus far, the literature on using social media to create desirable societal benefits, such as civic awareness and engagement, is still in its infancy. One key challenge in this research stream is to timely and accurately distill information from noisy social media data streams to community members. In this work, we develop SHEDR (social media–based hyperlocal event detection and recommendation), an end-to-end neural event detection and recommendation framework with a particular use case for Twitter to facilitate residents’ information seeking of hyperlocal events. The key model innovation in SHEDR lies in the design of the hyperlocal event detector and the event recommender. First, we harness the power of two popular deep neural network models, the convolutional neural network (CNN) and long short-term memory (LSTM), in a novel joint CNN-LSTM model to characterize spatiotemporal dependencies for capturing unusualness in a region of interest, which is classified as a hyperlocal event. Next, we develop a neural pairwise ranking algorithm for recommending detected hyperlocal events to residents based on their interests. To alleviate the sparsity issue and improve personalization, our algorithm incorporates several types of contextual information covering topic, social, and geographical proximities. We perform comprehensive evaluations based on two large-scale data sets comprising geotagged tweets covering Seattle and Chicago. We demonstrate the effectiveness of our framework in comparison with several state-of-the-art approaches. We show that our hyperlocal event detection and recommendation models consistently and significantly outperform other approaches in terms of precision, recall, and F-1 scores. Summary of Contribution: In this paper, we focus on a novel and important, yet largely underexplored application of computing—how to improve civic engagement in local neighborhoods via local news sharing and consumption based on social media feeds. To address this question, we propose two new computational and data-driven methods: (1) a deep learning–based hyperlocal event detection algorithm that scans spatially and temporally to detect hyperlocal events from geotagged Twitter feeds; and (2) A personalized deep learning–based hyperlocal event recommender system that systematically integrates several contextual cues such as topical, geographical, and social proximity to recommend the detected hyperlocal events to potential users. We conduct a series of experiments to examine our proposed models. The outcomes demonstrate that our algorithms are significantly better than the state-of-the-art models and can provide users with more relevant information about the local neighborhoods that they live in, which in turn may boost their community engagement.

Dimensionality reduction in the context of dynamic social media data streams

In recent years social media became an important part of everyday life for many people. A big challenge of social media is, to find posts, that are interesting for the user. Many social networks like Twitter handle this problem with so-called hashtags. A user can label his own Tweet (post) with a hashtag, while other users can search for posts containing a specified hashtag. But what about finding posts which are not labeled by the creator? We provide a way of completing hashtags for unlabeled posts using classification on a novel real-world Twitter data stream. New posts will be created every second, thus this context fits perfectly for non-stationary data analysis. Our goal is to show, how labels (hashtags) of social media posts can be predicted by stream classifiers. In particular, we employ random projection (RP) as a preprocessing step in calculating streaming models. Also, we provide a novel real-world data set for streaming analysis called NSDQ with a comprehensive data description. We show that this dataset is a real challenge for state-of-the-art stream classifiers. While RP has been widely used and evaluated in stationary data analysis scenarios, non-stationary environments are not well analyzed. In this paper, we provide a use case of RP on real-world streaming data, especially on NSDQ dataset. We discuss why RP can be used in this scenario and how it can handle stream-specific situations like concept drift. We also provide experiments with RP on streaming data, using state-of-the-art stream classifiers like adaptive random forest and concept drift detectors. Additionally, we experimentally evaluate an online principal component analysis (PCA) approach in the same fashion as we do for RP. To obtain higher dimensional synthetic streams, we use random Fourier features (RFF) in an online manner which allows us, to increase the number of dimensions of low dimensional streams.

Capturing Real-Time Public Space Activity Using Publicly Available Digital Traces

The study of public space activity has been one of the main foci in the debate on urban space transformations for the past decades, with many researchers adding to the debate through theoretical work, as well as empirical/quantitative evidence, drawing from their direct observations of public space activity. This paper attempts to enhance this approach of urban research and public space observation, by investigating the application of remote sensing techniques in public space analysis. More specifically, it attempts to capture public space activity using publicly available digital traces, such as environmental and temporal data, as well as social media data streams. By applying bivariate and multivariate analysis techniques to these datasets, it illustrates the possibility of capturing current activity in public spaces, with some degree of confidence. Furthermore, given the ubiquitous and real-time nature of these datasets, it also becomes possible to provide continuous estimates as well as short-term predictions on current and near-future public space use. Finally, it outlines the capabilities of this approach, to be used in complementary fashion to direct observation methods mentioned above, in building high resolution models and simulations of public space activity.

Real-time spatio-temporal event detection on geotagged social media

A key challenge in mining social media data streams is to identify events which are actively discussed by a group of people in a specific local or global area. Such events are useful for early warning for accident, protest, election or breaking news. However, neither the list of events nor the resolution of both event time and space is fixed or known beforehand. In this work, we propose an online spatio-temporal event detection system using social media that is able to detect events at different time and space resolutions. First, to address the challenge related to the unknown spatial resolution of events, a quad-tree method is exploited in order to split the geographical space into multiscale regions based on the density of social media data. Then, a statistical unsupervised approach is performed that involves Poisson distribution and a smoothing method for highlighting regions with unexpected density of social posts. Further, event duration is precisely estimated by merging events happening in the same region at consecutive time intervals. A post processing stage is introduced to filter out events that are spam, fake or wrong. Finally, we incorporate simple semantics by using social media entities to assess the integrity, and accuracy of detected events. The proposed method is evaluated using different social media datasets: Twitter and Flickr for different cities: Melbourne, London, Paris and New York. To verify the effectiveness of the proposed method, we compare our results with two baseline algorithms based on fixed split of geographical space and clustering method. For performance evaluation, we manually compute recall and precision. We also propose a new quality measure named strength index, which automatically measures how accurate the reported event is.

Embed2Detect: temporally clustered embedded words for event detection in social media

Social media is becoming a primary medium to discuss what is happening around the world. Therefore, the data generated by social media platforms contain rich information which describes the ongoing events. Further, the timeliness associated with these data is capable of facilitating immediate insights. However, considering the dynamic nature and high volume of data production in social media data streams, it is impractical to filter the events manually and therefore, automated event detection mechanisms are invaluable to the community. Apart from a few notable exceptions, most previous research on automated event detection have focused only on statistical and syntactical features in data and lacked the involvement of underlying semantics which are important for effective information retrieval from text since they represent the connections between words and their meanings. In this paper, we propose a novel method termed Embed2Detect for event detection in social media by combining the characteristics in word embeddings and hierarchical agglomerative clustering. The adoption of word embeddings gives Embed2Detect the capability to incorporate powerful semantical features into event detection and overcome a major limitation inherent in previous approaches. We experimented our method on two recent real social media data sets which represent the sports and political domain and also compared the results to several state-of-the-art methods. The obtained results show that Embed2Detect is capable of effective and efficient event detection and it outperforms the recent event detection methods. For the sports data set, Embed2Detect achieved 27% higher F-measure than the best-performed baseline and for the political data set, it was an increase of 29%.

A comprehensive social media data processing and analytics architecture by using big data platforms: a case study of twitter flood-risk messages.

The main objective of the article is to propose an advanced architecture and workflow based on Apache Hadoop and Apache Spark big data platforms. The primary purpose of the presented architecture is collecting, storing, processing, and analysing intensive data from social media streams. This paper presents how the proposed architecture and data workflow can be applied to analyse Tweets with a specific flood topic. The secondary objective, trying to describe the flood alert situation by using only Tweet messages and exploring the informative potential of such data is demonstrated as well. The predictive machine learning approach based on Bayes Theorem was utilized to classify flood and no flood messages. For this study, approximately 100,000 Twitter messages were processed and analysed. Messages were related to the flooding domain and collected over a period of 5 days (14 May – 18 May 2018). Spark application was developed to run data processing commands automatically and to generate the appropriate output data. Results confirmed the advantages of many well-known features of Spark and Hadoop in social media data processing. It was noted that such technologies are prepared to deal with social media data streams, but there are still challenges that one has to take into account. Based on the flood tweet analysis, it was observed that Twitter messages with some considerations are informative enough to be used to estimate general flood alert situations in particular regions. Text analysis techniques proved that Twitter messages contain valuable flood-spatial information.