Twitter Stream Research Articles

Social Noise: Generating Random Numbers from Twitter Streams

Due to the multiple applications of random numbers in computer systems (cryptography, online gambling, computer simulation, etc.) it is important to have mechanisms to generate these numbers. True Random Number Generators (TRNGs) are commonly used for this purpose. TRNGs rely on non-deterministic sources to generate randomness. Physical processes (like noise in semiconductors, quantum phenomenon, etc.) play this role in state of the art TRNGs. In this paper, we depart from previous work and explore the possibility of defining social TRNGs using the stream of public messages of the microblogging service Twitter as randomness source. Thus, we define two TRNGs based on Twitter stream information and evaluate them using the National Institute of Standards and Technology (NIST) statistical test suite. The results of the evaluation confirm the feasibility of the proposed approach.

Integrating Social and Auxiliary Semantics for Multifaceted Topic Modeling in Twitter

Microblogging platforms, such as Twitter, have already played an important role in recent cultural, social and political events. Discovering latent topics from social streams is therefore important for many downstream applications, such as clustering, classification or recommendation. However, traditional topic models that rely on the bag-of-words assumption are insufficient to uncover the rich semantics and temporal aspects of topics in Twitter. In particular, microblog content is often influenced by external information sources, such as Web documents linked from Twitter posts, and often focuses on specific entities, such as people or organizations. These external sources provide useful semantics to understand microblogs and we generally refer to these semantics as auxiliary semantics . In this article, we address the mentioned issues and propose a unified framework for Multifaceted Topic Modeling from Twitter streams. We first extract social semantics from Twitter by modeling the social chatter associated with hashtags. We further extract terms and named entities from linked Web documents to serve as auxiliary semantics during topic modeling. The Multifaceted Topic Model (MfTM) is then proposed to jointly model latent semantics among the social terms from Twitter, auxiliary terms from the linked Web documents and named entities. Moreover, we capture the temporal characteristics of each topic. An efficient online inference method for MfTM is developed, which enables our model to be applied to large-scale and streaming data. Our experimental evaluation shows the effectiveness and efficiency of our model compared with state-of-the-art baselines. We evaluate each aspect of our framework and show its utility in the context of tweet clustering.

Tweeting the Jihad: Social Media Networks of Western Foreign Fighters in Syria and Iraq

Social media have played an essential role in the jihadists’ operational strategy in Syria and Iraq, and beyond. Twitter in particular has been used to drive communications over other social media platforms. Twitter streams from the insurgency may give the illusion of authenticity, as a spontaneous activity of a generation accustomed to using their cell phones for self-publication, but to what extent is access and content controlled? Over a period of three months, from January through March 2014, information was collected from the Twitter accounts of 59 Western-origin fighters known to be in Syria. Using a snowball method, the 59 starter accounts were used to collect data about the most popular accounts in the network-at-large. Social network analysis on the data collated about Twitter users in the Western Syria-based fighters points to the controlling role played by feeder accounts belonging to terrorist organizations in the insurgency zone, and by Europe-based organizational accounts associated with the banned British organization, Al Muhajiroun, and in particular the London-based preacher, Anjem Choudary.

A case study of the New York City 2012-2013 influenza season with daily geocoded Twitter data from temporal and spatiotemporal perspectives.

BackgroundTwitter has shown some usefulness in predicting influenza cases on a weekly basis in multiple countries and on different geographic scales. Recently, Broniatowski and colleagues suggested Twitter’s relevance at the city-level for New York City. Here, we look to dive deeper into the case of New York City by analyzing daily Twitter data from temporal and spatiotemporal perspectives. Also, through manual coding of all tweets, we look to gain qualitative insights that can help direct future automated searches.ObjectiveThe intent of the study was first to validate the temporal predictive strength of daily Twitter data for influenza-like illness emergency department (ILI-ED) visits during the New York City 2012-2013 influenza season against other available and established datasets (Google search query, or GSQ), and second, to examine the spatial distribution and the spread of geocoded tweets as proxies for potential cases.MethodsFrom the Twitter Streaming API, 2972 tweets were collected in the New York City region matching the keywords “flu”, “influenza”, “gripe”, and “high fever”. The tweets were categorized according to the scheme developed by Lamb et al. A new fourth category was added as an evaluator guess for the probability of the subject(s) being sick to account for strength of confidence in the validity of the statement. Temporal correlations were made for tweets against daily ILI-ED visits and daily GSQ volume. The best models were used for linear regression for forecasting ILI visits. A weighted, retrospective Poisson model with SaTScan software (n=1484), and vector map were used for spatiotemporal analysis.ResultsInfection-related tweets (R=.763) correlated better than GSQ time series (R=.683) for the same keywords and had a lower mean average percent error (8.4 vs 11.8) for ILI-ED visit prediction in January, the most volatile month of flu. SaTScan identified primary outbreak cluster of high-probability infection tweets with a 2.74 relative risk ratio compared to medium-probability infection tweets at P=.001 in Northern Brooklyn, in a radius that includes Barclay’s Center and the Atlantic Avenue Terminal.ConclusionsWhile others have looked at weekly regional tweets, this study is the first to stress test Twitter for daily city-level data for New York City. Extraction of personal testimonies of infection-related tweets suggests Twitter’s strength both qualitatively and quantitatively for ILI-ED prediction compared to alternative daily datasets mixed with awareness-based data such as GSQ. Additionally, granular Twitter data provide important spatiotemporal insights. A tweet vector-map may be useful for visualization of city-level spread when local gold standard data are otherwise unavailable.

Umbrella Protection Against Unauthorized Black Blanket Address on Public Webbing

Twitter is prone to malicious tweets containing URLs for spam, phishing, and malware distribution. Conventional Twitter spam detection schemes utilize account features such as the ratio of tweets containing URLs and the account creation date, or relation features in the Twitter graph. These detection schemes are ineffective against feature fabrications or consume much time and resources. Conventional suspicious URL detection schemes utilize several features including lexical features of URLs, URL redirection, HTML content, and dynamic behavior. However, evading techniques such as time-based evasion and crawler evasion exist. In this paper, we propose WARNINGBIRD, a suspicious URL detection system for Twitter. Our system investigates correlations of URL redirect chains extracted from several tweets. Because attackers have limited resources and usually reuse them, their URL redirect chains frequently share the same URLs. We develop methods to discover correlated URL redirect chains using the frequently shared URLs and to determine their suspiciousness. We collect numerous tweets from the Twitter public timeline and build a statistical classifier using them. Evaluation results show that our classifier accurately and efficiently detects suspicious URLs. WARNINGBIRD as a near real-time system for classifying suspicious URLs in the Twitter stream. In this project I proposed block the malicious URLs and provide mail alert for malicious URLs occur in the twitter stream.

Non-Parametric Scan Statistics for Disease Outbreak Detection on Twitter

We present a new method for disease outbreak detection, the "Non-Parametric Heterogeneous Graph Scan (NPHGS)". NPHGS enables fast and accurate detection of emerging space-time clusters using Twitter and other social media streams where standard parametric model assumptions are incorrect.

Estimating the Locations of Emergency Events from Twitter Streams

Sina Weibo, the most popular microblogging service in China, plays a significance role in sharing and exchanging information on local and global level. Actually, part of the information are correlated with some events which may be local ones such as accident, protests or natural disasters to widely ones such as events concerning well-known people or political affairs. Our research interest is to detect the location where some events are occurring by analyzing only data from weibo information. In this paper, we de- note three kind of location for each weibo: (1) content-based location; (2) posting location; (3) registration location of author of weibo. There are two main contributions in our work: (a) detect event-aware location of each weibo; (b) estimate accurate event location based on sets of event-aware locations. In order to detect event-aware location, we model there kinds of location data of each weibo to tackle with detection challenge. After we obtained a set of event-aware locations, we use a hierarchical clustering to estimate the accurate event location. The result showed that our method could improve the accuracy of event location estimation.

Tracking Suicide Risk Factors Through Twitter in the US

Suicide is a leading cause of death in the United States. Social media such as Twitter is an emerging surveillance tool that may assist researchers in tracking suicide risk factors in real time. To identify suicide-related risk factors through Twitter conversations by matching on geographic suicide rates from vital statistics data. At-risk tweets were filtered from the Twitter stream using keywords and phrases created from suicide risk factors. Tweets were grouped by state and departures from expectation were calculated. The values for suicide tweeters were compared against national data of actual suicide rates from the Centers for Disease Control and Prevention. A total of 1,659,274 tweets were analyzed over a 3-month period with 37,717 identified as at-risk for suicide. Midwestern and western states had a higher proportion of suicide-related tweeters than expected, while the reverse was true for southern and eastern states. A strong correlation was observed between state Twitter-derived data and actual state age-adjusted suicide data. Twitter may be a viable tool for real-time monitoring of suicide risk factors on a large scale. This study demonstrates that individuals who are at risk for suicide may be detected through social media.

The Political Domain Goes to Twitter: Hashtags, Retweets and URLs

The argument is twofold. One, the character of political communication on Twitter is sufficiently different from the general character of the Twitter stream, from the “firehose”, as it is known that political communication should be considered as a separable domain of communication. Specifically, retweets, urls, and hashtags are used far more frequently in political communication than is true for the full stream of messages and that reflects communication which is more interactive than is generally the case. Two, context is needed for characterizing twitter streams. In the case of political communication there are parameters on the use of these tools, which facilitate interactive communication that sets such a context. If most political tweets have retweets or urls between some low bound and a high bound, then one has a way to characterize any specific stream that is being investigated. The analyses will begin the investigation of these parameters.

Enacting identity in microblogging through ambient affiliation

This article explores how we use social media to construe identities and align with others into communities of shared values. The focus is on how ‘users of language perform their identities within uses of language’: How do personae using the microblogging service Twitter perform relational identities as they enact discourse fellowships? Addressing this question means understanding how personae enter into ambient affiliation. Such affiliation is ambient in the sense that social media users may not be interacting directly, but instead participating in mass performances of hashtagging or contributing to iterations of Internet memes. This article will consider three key bonds ( self-deprecation, frazzle and addiction) using both a 100 million-word corpus of posts and a smaller specialized corpus collected by capturing the entire Twitter stream of a particular user.

ScatterBlogs2: Real-Time Monitoring of Microblog Messages through User-Guided Filtering

The number of microblog posts published daily has reached a level that hampers the effective retrieval of relevant messages, and the amount of information conveyed through services such as Twitter is still increasing. Analysts require new methods for monitoring their topic of interest, dealing with the data volume and its dynamic nature. It is of particular importance to provide situational awareness for decision making in time-critical tasks. Current tools for monitoring microblogs typically filter messages based on user-defined keyword queries and metadata restrictions. Used on their own, such methods can have drawbacks with respect to filter accuracy and adaptability to changes in trends and topic structure. We suggest ScatterBlogs2, a new approach to let analysts build task-tailored message filters in an interactive and visual manner based on recorded messages of well-understood previous events. These message filters include supervised classification and query creation backed by the statistical distribution of terms and their co-occurrences. The created filter methods can be orchestrated and adapted afterwards for interactive, visual real-time monitoring and analysis of microblog feeds. We demonstrate the feasibility of our approach for analyzing the Twitter stream in emergency management scenarios.

Online Survey on Twitter: A Urological Experience

After my return from the Annual Congress of the American Urological Association in San Diego (May 3-8, 2013), I would like to congratulate McKendrick et al [1] and Hanson et al [2] on their recent papers on the impact of Twitter, one of the most common social media, in the medical and non-medical fields. In their elegant communications, both groups of authors stressed the role of the Internet and the Web as new methods of publicizing scientific data. Starting from the use of Twitter during a medical conference, McKendrick et al reported their first experience of using social media as supports for conference organizers, highlighting the use of a Twitter stream as an integral part of the communication structure of a conference on anesthetics. Hanson et al reported a Twitter-based surveillance method for monitoring public health regarding the use and abuse of a psychostimulant drug, emphasizing the potential role of this social media in collecting data for a survey. To confirm these findings and encourage the use of these novel tools “to do science”, I would like to share a personal pioneering experiment I carried out during the Annual Congress of the European Association of Urology (March 2013, in Milan) and the recent American Congress in San Diego. During the meetings, I launched an online survey using Twitter, and posted an interactive specific question concerning the choice of preferred approach to robotic radical prostatectomy, the most recent and widespread surgical intervention for prostate cancer. The tweet was: “ONLINE-SURVEY: Do you perform ONLY Transperitoneal (T), ONLY Extraperitoneal (E) or BOTH approaches to dVP? (dVP= Da Vinci robotic prostatectomy) ReTweet T, E or TE” (Figure 1). I used two specific hashtags (#EAU2013 for Milan, and #AUA2013 for San Diego). In the course of both congresses, I received no fewer than 326 answers. Considering the absolute number of participants (approximately 18,000 in both congresses) the percentage of Tweets received (about 2%) may seem very low, but considering that it was a first urological experience, I think it may be significant. Evaluating the survey results, 81 out of the 326 urologists (24.9% of those who used Twitter and retweeted me their answers!) perform both procedures. In 165 cases (50.6%), the preferred approach to robotic prostatectomy is transperitoneal. I presented the results of this survey during my podium lecture in San Diego. Obviously, my study is not a scientific survey without biases, but it did reveal the spread of social media, to the scientific community too. In my opinion, this simple experiment not only confirms your own considerations about the use of social media, but also helps all of us to appreciate the fact that Twitter, like Facebook, Google+, Linkedin and all the other Web-linked social tools, is not limited to young people “chatting about love or friends”, but are novel instruments which can be used to even greater scientific advantage. [J Med Internet Res 2013;15(10):e238]

A Survey of Techniques for Event Detection in Twitter

Twitter is among the fastest‐growing microblogging and online social networking services. Messages posted on Twitter (tweets) have been reporting everything from daily life stories to the latest local and global news and events. Monitoring and analyzing this rich and continuous user‐generated content can yield unprecedentedly valuable information, enabling users and organizations to acquire actionable knowledge. This article provides a survey of techniques for event detection from Twitter streams. These techniques aim at finding real‐world occurrences that unfold over space and time. In contrast to conventional media, event detection from Twitter streams poses new challenges. Twitter streams contain large amounts of meaningless messages and polluted content, which negatively affect the detection performance. In addition, traditional text mining techniques are not suitable, because of the short length of tweets, the large number of spelling and grammatical errors, and the frequent use of informal and mixed language. Event detection techniques presented in literature address these issues by adapting techniques from various fields to the uniqueness of Twitter. This article classifies these techniques according to the event type, detection task, and detection method and discusses commonly used features. Finally, it highlights the need for public benchmarks to evaluate the performance of different detection approaches and various features.

Towards social data platform

Many novel applications have been built based on analyzing tweets about specific topics. While these applications provide different kinds of analysis, they share a common task of monitoring "target" tweets from the Twitter stream for a topic. The current solution for this task tracks a set of manually selected keywords with Twitter APIs. Obviously, this manual approach has many limitations. In this paper, we propose a data platform to automatically monitor target tweets from the Twitter stream for any given topic. To monitor target tweets in an optimal and continuous way, we design Automatic Topic-focused Monitor (ATM), which iteratively 1) samples tweets from the stream and 2) selects keywords to track based on the samples. To realize ATM, we develop a tweet sampling algorithm to sample sufficient unbiased tweets with available Twitter APIs, and a keyword selection algorithm to efficiently select keywords that have a near-optimal coverage of target tweets under cost constraints. We conduct extensive experiments to show the effectiveness of ATM. E.g., ATM covers 90% of target tweets for a topic and improves the manual approach by 49%.

EvenTweet

Microblogging services such as Twitter, Facebook, and Foursquare have become major sources for information about real-world events. Most approaches that aim at extracting event information from such sources typically use the temporal context of messages. However, exploiting the location information of georeferenced messages, too, is important to detect localized events, such as public events or emergency situations. Users posting messages that are close to the location of an event serve as human sensors to describe an event. In this demonstration, we present a novel framework to detect localized events in real-time from a Twitter stream and to track the evolution of such events over time. For this, spatio-temporal characteristics of keywords are continuously extracted to identify meaningful candidates for event descriptions. Then, localized event information is extracted by clustering keywords according to their spatial similarity. To determine the most important events in a (recent) time frame, we introduce a scoring scheme for events. We demonstrate the functionality of our system, called Even-Tweet, using a stream of tweets from Europe during the 2012 UEFA European Football Championship.

Use of Hangeul Twitter to Track and Predict Human Influenza Infection

Influenza epidemics arise through the accumulation of viral genetic changes. The emergence of new virus strains coincides with a higher level of influenza-like illness (ILI), which is seen as a peak of a normal season. Monitoring the spread of an epidemic influenza in populations is a difficult and important task. Twitter is a free social networking service whose messages can improve the accuracy of forecasting models by providing early warnings of influenza outbreaks. In this study, we have examined the use of information embedded in the Hangeul Twitter stream to detect rapidly evolving public awareness or concern with respect to influenza transmission and developed regression models that can track levels of actual disease activity and predict influenza epidemics in the real world. Our prediction model using a delay mode provides not only a real-time assessment of the current influenza epidemic activity but also a significant improvement in prediction performance at the initial phase of ILI peak when prediction is of most importance.

Collection and visualization of dietary behavior and reasons for eating using Twitter.

BackgroundIncreasing an individual’s awareness and understanding of their dietary habits and reasons for eating may help facilitate positive dietary changes. Mobile technologies allow individuals to record diet-related behavior in real time from any location; however, the most popular software applications lack empirical evidence supporting their efficacy as health promotion tools.ObjectiveThe purpose of this study was to test the feasibility and acceptability of a popular social media software application (Twitter) to capture young adults’ dietary behavior and reasons for eating. A secondary aim was to visualize data from Twitter using a novel analytic tool designed to help identify relationships among dietary behaviors, reasons for eating, and contextual factors.MethodsParticipants were trained to record all food and beverages consumed over 3 consecutive days (2 weekdays and 1 weekend day) using their mobile device’s native Twitter application. A list of 24 hashtags (#) representing food groups and reasons for eating were provided to participants to guide reporting (eg, #protein, #mood). Participants were encouraged to annotate hashtags with contextual information using photos, text, and links. User experience was assessed through a combination of email reports of technical challenges and a 9-item exit survey. Participant data were captured from the public Twitter stream, and frequency of hashtag occurrence and co-occurrence were determined. Contextual data were further parsed and qualitatively analyzed. A frequency matrix was constructed to identify food and behavior hashtags that co-occurred. These relationships were visualized using GMap algorithmic mapping software.ResultsA total of 50 adults completed the study. In all, 773 tweets including 2862 hashtags (1756 foods and 1106 reasons for eating) were reported. Frequently reported food groups were #grains (n=365 tweets), #dairy (n=221), and #protein (n=307). The most frequently cited reasons for eating were #social (activity) (n=122), #taste (n=146), and #convenience (n=173). Participants used a combination of study-provided hash tags and their own hash tags to describe behavior. Most rated Twitter as easy to use for the purpose of reporting diet-related behavior. “Maps” of hash tag occurrences and co-occurrences were developed that suggested time-varying diet and behavior patterns.ConclusionsTwitter combined with an analytical software tool provides a method for capturing real-time food consumption and diet-related behavior. Data visualization may provide a method to identify relationships between dietary and behavioral factors. These findings will inform the design of a study exploring the use of social media and data visualization to identify relationships between food consumption, reasons for engaging in specific food-related behaviors, relevant contextual factors, and weight and health statuses in diverse populations.

WarningBird: A Near Real-Time Detection System for Suspicious URLs in Twitter Stream

Twitter is prone to malicious tweets containing URLs for spar, phishing, and malware distribution. Conventional Twitter spar detection schemes utilize account features such as the ratio of tweets containing URLs and the account creation date, or relation features in the Twitter graph. These detection schemes are ineffective against feature fabrications or consume much time and resources. Conventional suspicious URL detection schemes utilize several features including lexical features of URLs, URL redirection, HTIUIL content, and dynamic behavior. However, evading techniques such as time-based evasion and crawler evasion exist. in this paper, we propose WARNINGBIRD, a suspicious URL detection system for Twitter. Our system investigates correlations of URL redirect chains extracted from several tweets. Because attackers have limited resources and usually reuse them, their URL redirect chains frequently share the same URLs. We develop methods to discover correlated URL redirect chains using the frequently shared URLs and to determine their suspiciousness. We collect numerous tweets from the Twitter public timeline and build a statistical classifier using them. Evaluation results show that our classifier accurately and efficiently detects suspicious URLs. We also present WARNINGBIRD as a near real-time system for classifying suspicious URLs in the Twitter stream.

Public reasoning around social contention: A case study of Twitter use in the Italian mobilization for global change

Social media have become central to the organizational and logistic dynamics in contemporary social contention. Not only have they eased centralized processes of information sharing between social movement entrepreneurs and social movement publics, they have also bolstered the crowdsourced management of public discussion around social contention. This article investigates the use of social media for public reasoning around issues related to social contention. By investigating Twitter streams with reference to the Italian chapter of the 15 October 2011 polycentric protest for global change, this study specifically addresses the use of this medium in the aftermath of protest events of high impact for the general public. The quantitative analysis of longitudinal progression, networking mechanisms and processes of meaning construction in over 8000 tweets shows that Twitter does bolster public reasoning around social contention. It does so as a news medium rather than a conversational platform, primarily providing a space to share information alternative to that available in mainstream media coverage.

Achieving the CSHP’s Vision for Hospital Pharmacy in Canada

to the changing needs of members for communication, participation, and learning by building and expanding digital communication channels. You can join CSHP on Facebook, participate in CSHP 2015 blogs and Twitter streams, and take advantage of the topical education webinars. During the PPC, check out award-winning residency projects, virtual posters, and student video competition entries, and consider participating in next year’s awards programs. You can also connect with and learn from colleagues by joining one of CSHP’s 23 Pharmacy Specialty Networks (PSNs), which cover diverse areas such as antimicrobial stewardship, drug utilization, emergency, home care, pediatrics, and transplantation. There is even a PSN specifically for pharmacists practising in small hospitals. Participating members rate PSNs as informative with regard to new practices and guidelines, ideas on how to better serve patients and share knowledge, connecting with peers across Canada, and more. What better way to connect efficiently and productively with colleagues with similar interests and expertise that they are willing to share? Check www.cshp.ca for information. Pharmacists are creative, innovative, and dedicated professionals, keen to collaborate to advance patient care. I encourage you to take the initiative: participate, lead, be engaged, and promote our profession.