Social Media Corpus Research Articles

Large Language Models Can Enable Inductive Thematic Analysis of a Social Media Corpus in a Single Prompt: Human Validation Study.

Manually analyzing public health-related content from social media provides valuable insights into the beliefs, attitudes, and behaviors of individuals, shedding light on trends and patterns that can inform public understanding, policy decisions, targeted interventions, and communication strategies. Unfortunately, the time and effort needed from well-trained human subject matter experts makes extensive manual social media listening unfeasible. Generative large language models (LLMs) can potentially summarize and interpret large amounts of text, but it is unclear to what extent LLMs can glean subtle health-related meanings in large sets of social media posts and reasonably report health-related themes. We aimed to assess the feasibility of using LLMs for topic model selection or inductive thematic analysis of large contents of social media posts by attempting to answer the following question: Can LLMs conduct topic model selection and inductive thematic analysis as effectively as humans did in a prior manual study, or at least reasonably, as judged by subject matter experts? We asked the same research question and used the same set of social media content for both the LLM selection of relevant topics and the LLM analysis of themes as was conducted manually in a published study about vaccine rhetoric. We used the results from that study as background for this LLM experiment by comparing the results from the prior manual human analyses with the analyses from 3 LLMs: GPT4-32K, Claude-instant-100K, and Claude-2-100K. We also assessed if multiple LLMs had equivalent ability and assessed the consistency of repeated analysis from each LLM. The LLMs generally gave high rankings to the topics chosen previously by humans as most relevant. We reject a null hypothesis (P<.001, overall comparison) and conclude that these LLMs are more likely to include the human-rated top 5 content areas in their top rankings than would occur by chance. Regarding theme identification, LLMs identified several themes similar to those identified by humans, with very low hallucination rates. Variability occurred between LLMs and between test runs of an individual LLM. Despite not consistently matching the human-generated themes, subject matter experts found themes generated by the LLMs were still reasonable and relevant. LLMs can effectively and efficiently process large social media-based health-related data sets. LLMs can extract themes from such data that human subject matter experts deem reasonable. However, we were unable to show that the LLMs we tested can replicate the depth of analysis from human subject matter experts by consistently extracting the same themes from the same data. There is vast potential, once better validated, for automated LLM-based real-time social listening for common and rare health conditions, informing public health understanding of the public's interests and concerns and determining the public's ideas to address them.

People believe political opponents accept blatant moral wrongs, fueling partisan divides.

Efforts to bridge political divides often focus on navigating complex and divisive issues, but eight studies reveal that we should also focus on a more basic misperception: that political opponents are willing to accept basic moral wrongs. In the United States, Democrats, and Republicans overestimate the number of political outgroup members who approve of blatant immorality (e.g. child pornography, embezzlement). This "basic morality bias" is tied to political dehumanization and is revealed by multiple methods, including natural language analyses from a large social media corpus and a survey with a representative sample of Americans. Importantly, the basic morality bias can be corrected with a brief, scalable intervention. Providing information that just one political opponent condemns blatant wrongs increases willingness to work with political opponents and substantially decreases political dehumanization.

Hebrew Loanwords in Two Rural Dialects of Arabic in Israel

Abstract This study examines loanwords from Hebrew into Arabic in the spoken language of people from Iksal village, in the lower Galilee and Um Al-Fahm city, in the Triangle region in Israel. The current study specifically examines borrowing in spoken language, with a comparison of the current findings with a previous research by Abu Elhija (2017), where the data was taken from online writings on Facebook. The research questions the frequency of loanword items from Hebrew into two rural dialects, the domains that appear with Hebrew loanwords, the phonological representations of the loanwords, and the reasons for these words to be borrowed. The main findings show that more loanwords were found in the spoken corpus than in the social media corpus in Abu Elhija (2017). There are more loanwords in synchronous communication compared to asynchronous. The findings also show that topics like education, employment, and technology had the most extensive loanwords. Moreover, in the case of loanwords used by Arabs in Israel, a word can be considered a loanword regardless of its phonology.

Overview of the 8th Social Media Mining for Health Applications (#SMM4H) shared tasks at the AMIA 2023 Annual Symposium.

The aim of the Social Media Mining for Health Applications (#SMM4H) shared tasks is to take a community-driven approach to address the natural language processing and machine learning challenges inherent to utilizing social media data for health informatics. In this paper, we present the annotated corpora, a technical summary of participants' systems, and the performance results. The eighth iteration of the #SMM4H shared tasks was hosted at the AMIA 2023 Annual Symposium and consisted of 5 tasks that represented various social media platforms (Twitter and Reddit), languages (English and Spanish), methods (binary classification, multi-class classification, extraction, and normalization), and topics (COVID-19, therapies, social anxiety disorder, and adverse drug events). In total, 29 teams registered, representing 17 countries.In general, the top-performing systems used deep neural network architectures based on pre-trained transformer models. In particular, the top-performing systems for the classification tasks were based on single models that were pre-trained on social media corpora. To facilitate future work, the datasets-a total of 61353 posts-will remain available by request, and the CodaLab sites will remain active for a post-evaluation phase.

Design and construction of a social media corpus: Influencers’ speech in vlogs

This article outlines the creation of a social media corpus of Turkish vlogs on YouTube, aimed at analyzing the translanguaging practices and multimodal communication of Turkish social media influencers. It firstly describes the process of constructing the corpus, including transcription conventions and ad hoc annotation. The article then analyzes the phenomenon of translanguaging, with an emphasis on its prevalent forms and modes. Given the challenges associated with compiling a multimodally rich social media corpus, this paper provides strategies for manually transcribing and annotating linguistic and semiotic features in ELAN software, as well as strategies for managing tier-based annotations for vlog datasets. Additionally, the study presents approaches for handling non-standard linguistic codes and marked occurrences in language contact zones, illustrated through examples drawn from the vlog corpus where Turkish serves as the standard code.

Translanguaging dynamics in the digital landscape: insights from a social media corpus

Based on a new Turkish social media influencer corpus consisting of 30 YouTube vlogs, this study explores young adults’ translanguaging practices, prevalent linguistic events, and spatial repertoires in communication. The analysis focuses on vlogs, where linguistic resources become available in relation to the activities, audience, and organization of places or objects and young adults’ translanguaging practices and their situated functions. The findings demonstrate how to use translanguaging as a comprehensive methodology to analyze communication unfolding with multimodal, multilingual, and digital semiotic repertoires at speakers’ disposal and for the imagined audience integral to the communication in vlogs. Taking a corpus-linguistics perspective to translanguaging, we propose a usage-based focus and undertake a broader understanding of translanguaging as a linguistic and multimodal phenomenon. ABSTRACT (TURKISH) 30 YouTube vlogundan oluşan yeni bir Türk sosyal medya etkileyicileri derlemine dayanan bu çalışma, genç yetişkinlerin diller ötesi pratiklerini, vloglarda hakim olan dilsel olayları ve iletişimdeki mekansal repertuarlarını incelemektedir. Analizin odağında genç yetişkinlerin dilsel kaynaklarının, etkinlikler, izleyici ve mekanların veya nesnelerin düzenlenmesi, diller ötesi pratikler ve bu pratiklerin yerleşik işlevleri ile ilişkili olarak kullanılabilir hale geldiği vloglar bulunmaktadır. Bulgular, vloglardaki iletişimin ayrılmaz bir parçası olan hayali izleyici için üretilen iletişimin konuşmacıların tasarrufunda olan çoklu modlu, çokdilli ve dijital göstergesel repertuarlar ile bir bütün olduğunu ve bu iletişimi çözümlemek için kapsamlı bir metodoloji olan diller ötesilik kuramının kullanılabileceğini göstermektedir. Bu çalışma, diller ötesiliği derlembilim destekli bir yaklaşımla ele alarak, kullanıma dayalı bir odak önermekte ve diller ötesiliği dilbilimsel ve çok modlu bir olgu olarak daha kapsamlı bir anlayışla ele almaktadır.

A New Social Media Analytics Method for Identifying Factors Contributing to COVID-19 Discussion Topics

Since the onset of the COVID-19 crisis, scholarly investigations and policy formulation have harnessed the potent capabilities of artificial intelligence (AI)-driven social media analytics. Evidence-driven policymaking has been facilitated through the proficient application of AI and natural language processing (NLP) methodologies to analyse the vast landscape of social media discussions. However, recent research works have failed to demonstrate a methodology to discern the underlying factors influencing COVID-19-related discussion topics. In this scholarly endeavour, an innovative AI- and NLP-based framework is deployed, incorporating translation, sentiment analysis, topic analysis, logistic regression, and clustering techniques to meticulously identify and elucidate the factors that are relevant to any discussion topics within the social media corpus. This pioneering methodology is rigorously tested and evaluated using a dataset comprising 152,070 COVID-19-related tweets, collected between 15th July 2021 and 20th April 2023, encompassing discourse in 58 distinct languages. The AI-driven regression analysis revealed 37 distinct observations, with 20 of them demonstrating a higher level of significance. In parallel, clustering analysis identified 15 observations, including nine of substantial relevance. These 52 AI-facilitated observations collectively unveil and delineate the factors that are intricately linked to five core discussion topics that are prevalent in the realm of COVID-19 discourse on Twitter. To the best of our knowledge, this research constitutes the inaugural effort in autonomously identifying factors associated with COVID-19 discussion topics, marking a pioneering application of AI algorithms in this domain. The implementation of this method holds the potential to significantly enhance the practice of evidence-based policymaking pertaining to matters concerning COVID-19.

Hack your corpus analysis: How AI can assist corpus linguists deal with messy social media data

This short reflection considers the role of Artificial Intelligence (AI) language models in supporting the work of corpus linguists. The focus is on how AI can be used in the analysis of complex and 'noisy' data commonly found in social media corpora. These datasets are characterized by unconventional language usage, shorthand, hashtags, emoji, and various other forms of creative expression. The discussion will be grounded in the author's ongoing research into emoji which offers an interesting example of the challenges of working with AI in the context of digital discourse analysis. The primary focus will be on how ChatGPT can support the creation and implementation of regular expressions and code for searching, matching, and manipulating text in corpora. The main objective is to guide the reader through the author's reasoning process as a way of reflecting on the integration of AI technology into the academic practice of corpus linguists.

Strategies for the Analysis of Large Social Media Corpora: Sampling and Keyword Extraction Methods

In the context of the COVID-19 pandemic, social media platforms such as Twitter have been of great importance for users to exchange news, ideas, and perceptions. Researchers from fields such as discourse analysis and the social sciences have resorted to this content to explore public opinion and stance on this topic, and they have tried to gather information through the compilation of large-scale corpora. However, the size of such corpora is both an advantage and a drawback, as simple text retrieval techniques and tools may prove to be impractical or altogether incapable of handling such masses of data. This study provides methodological and practical cues on how to manage the contents of a large-scale social media corpus such as Chen et al. (JMIR Public Health Surveill 6(2):e19273, 2020) COVID-19 corpus. We compare and evaluate, in terms of efficiency and efficacy, available methods to handle such a large corpus. First, we compare different sample sizes to assess whether it is possible to achieve similar results despite the size difference and evaluate sampling methods following a specific data management approach to storing the original corpus. Second, we examine two keyword extraction methodologies commonly used to obtain a compact representation of the main subject and topics of a text: the traditional method used in corpus linguistics, which compares word frequencies using a reference corpus, and graph-based techniques as developed in Natural Language Processing tasks. The methods and strategies discussed in this study enable valuable quantitative and qualitative analyses of an otherwise intractable mass of social media data.

Characterization Frames Constructing Endoxa in Activists’ Discourse About the Public Controversy Surrounding Fashion Sustainability

This paper investigates the relationship between characterization frames and argumentation in activists’ discourse about the public controversy surrounding fashion sustainability. While previous studies proposing an argumentative approach to frames have acknowledged that frames are related to underlying implicit premises, how frames select certain implicit premises still needs to be systematically explained. Therefore, drawing on a theoretical framework combining Pragma dialectics (van Eemeren 2010) with the Argumentum Model of Topics an empirical analysis of a social media corpus has been performed in order to examine the connection between characterization frames and underlying implicit premises. This paper offers both a methodological and a theoretical contribution. From a methodological point of view, it offers linguistic-discursive tools for the analysis of characterization frames. From a theoretical perspective, it shows how characterization frames contribute to construct endoxa, i.e. underlying implicit premises, through argumentative patterns.

Detecting racial stereotypes: An Italian social media corpus where psychology meets NLP

The generation of stereotypes allows us to simplify the cognitive complexity we have to deal with in everyday life. Stereotypes are extensively used to describe people who belong to a different ethnic group, particularly in racial hoaxes and hateful content against immigrants. This paper addresses the study of stereotypes from a novel perspective that involves psychology and computational linguistics both. On the one hand, it describes an Italian social media corpus built within a social psychology study, where stereotypes and related forms of discredit were made explicit through annotation. On the other hand, it provides some lexical analysis, to bring out the linguistic features of the messages collected in the corpus, and experiments for validating this annotation scheme and its automatic application to other corpora in the future. The main expected outcome is to shed some light on the usefulness of this scheme for training tools that automatically detect and label stereotypes in Italian.

Mapping Digital Discourses of the Capital Region of Finland

This article discusses the three Finnish city names Helsinki, Espoo, and Vantaa, and the urban discourses that surround them. The study reveals patterns of socio-spatial differentiation by examining what meanings people attach to these capital region cities and investigating how these meanings are expressed in online discourses. Using the methodological approach of corpus-assisted onomastics (CAO), this study incorporates onomastics, geographical information systems (GIS), and corpus linguistics. This interdisciplinary research also examines how corpus-assisted discourse studies (CADS) and GIS can be combined to reveal and visualize the contextual information and discursive patterns of toponyms. For this investigation, a data set of 2.7 billion words was collected from the Suomi24 Corpus, one of the biggest discussion fora in Finland. The use of social media corpus as a data source increases the authenticity of this research, as the data was not collected specifically for this study. The analysis reveals that the most frequent digital discourses about the cities refer to places and directions, housing, and mobility. The occurrences differ quantitatively and qualitatively between each city. This paper paves the way for future onomastic studies to research actual name usage using this new methodology. Knowledge gained from such research may not only enrich the field of onomastics, but also facilitate more socially sustainable urban planning.

Public perception and usage of the term carbon: Linguistic analysis in an environmental social media corpus

Abstract The main objective of this study was to examine the specialized environmental vocabulary which is incorporated in ordinary discourse found on the Internet. Specifically, we collected English-language data from two social networks, Twitter and Reddit, with a set of environmental keywords which was compiled using the term candidates extraction technique described in Shvets and Wanner (2020). Furthermore, using data from the initial corpus, we built a smaller sample of texts to serve as a ground for a preliminary linguistic analysis of the environmental term carbon. In environmental discourse, carbon is an umbrella term which has acquired multiple senses when it comes to greenhouse gas emissions. Additionally, this term has settled in ordinary communication (Fletcher &amp; Downing, 2011). By means of a fine-grained manual linguistic analysis applied to the data sample, we identified five semantic patterns in the way the general public conceptualizes carbon in the environmental context.

CREDBANK: A Large-Scale Social Media Corpus With Associated Credibility Annotations

Social media has quickly risen to prominence as a news source, yet lingering doubts remain about its ability to spread rumor and misinformation. Systematically studying this phenomenon, however, has been difficult due to the need to collect large-scale, unbiased data along with in-situ judgements of its accuracy. In this paper we present CREDBANK, a corpus designed to bridge this gap by systematically combining machine and human computation. Specifically, CREDBANK is a corpus of tweets, topics, events and associated human credibility judgements. It is based on the real-time tracking of more than 1 billion streaming tweets over a period of more than three months, computational summarizations of those tweets, and intelligent routings of the tweet streams to human annotators — within a few hours of those events unfolding on Twitter. In total CREDBANK comprises more than 60 million tweets grouped into 1049 real-world events, each annotated by 30 human annotators. As an example, with CREDBANK one can quickly calculate that roughly 24% of the events in the global tweet stream are not perceived as credible. We have made CREDBANK publicly available, and hope it will enable new research questions related to online information credibility in fields such as social science, data mining and health.

Generate Adjective Sentiment Dictionary for Social Media Sentiment Analysis Using Constrained Nonnegative Matrix Factorization

Although sentiment analysis has attracted a lot of research, little work has been done on social media data compared to product and movie reviews. This is due to the low accuracy that results from the more informal writing seen in social media data. Currently, most of sentiment analysis tools on social media choose the lexicon-based approach instead of the machine learning approach because the latter requires the huge challenge of obtaining enough human-labeled training data for extremely large-scale and diverse social opinion data. The lexicon-based approach requires a sentiment dictionary to determine opinion polarity. This dictionary can also provide useful features for any supervised learning method of the machine learning approach. However, many benchmark sentiment dictionaries do not cover the many informal and spoken words used in social media. In addition, they are not able to update frequently to include newly generated words online. In this paper, we present an automatic sentiment dictionary generation method, called Constrained Symmetric Nonnegative Matrix Factorization (CSNMF) algorithm, to assign polarity scores to each word in the dictionary, on a large social media corpus — digg.com. Moreover, we will demonstrate our study of Amazon Mechanical Turk (AMT) on social media word polarity, using both the human-labeled dictionaries from AMT and the General Inquirer Lexicon to compare our generated dictionary with. In our experiment, we show that combining links from both WordNet and the corpus to generate sentiment dictionaries does outperform using only one of them, and the words with higher sentiment scores yield better precision. Finally, we conducted a lexicon-based sentiment analysis on human-labeled social comments using our generated sentiment dictionary to show the effectiveness of our method.

The use of hēi diào (‘to turn black’) and its related [V diào] forms in social media

Abstract Hēi diào ‘to turn black’ (黑掉) denotes a change-of-state meaning, but in social media, it has a special pragmatic use that emphasizes “bad consequences” following an uncooperative act, whether right or wrong. This metaphorical and new transitive use of hēi diào, which is unique among netizens, has changed the literal status of hēi ‘black’ to other additional accomplishment/achievement meanings. Our study examined the expression hēi diào using data from a social media corpus, the PTT Bulletin Board System. We also analysed all the constructions of hēi diào and similar [V diào (掉)] patterns in the corpus. Unlike most previous work, which analysed the uses of [V diào] only, our study observed pragmatic connotations of hēi diào in social media—uses that only materialized in certain contexts.

Information extraction from digital social trace data with applications to social media and scholarly communication data

Shubhanshu is a Machine Learning Researcher at Twitter working on the Content Understanding Research team. He finished his Ph.D. at the iSchool, University of Illinois at Urbana-Champaign, where he worked as a research assistant with Dr. Jana Diesner and Dr. Vetle Torvik on projects funded by NIH, NSF, KISTI, and Army Research Lab. Shubhanshu's research work is at the intersection of machine learning, information extraction, social network analysis, and visualizations. His work won then Best Student Paper Award at ASIS&amp;T SIGMET workshop. Shubhanshu has published multiple open source projects as well as open datasets related to his research. More information about Shubhanshu can be found at: https://shubhanshu.com. In his thesis, Shubhanshu worked on unifying the concept of information extraction from social media data and scholarly data under the framework of Digital Social Trace Data (DSTD). This framework allow us to answer questions like conceptual novelty, expertise in scientific communities as well as influential users in social media corpus. He further proposed improved information extraction techniques for social media text by using active, semi-supervised, and multi-task machine learning along with the inclusion of meta-data. Additionally, his work resulted in the development of a visualization framework for DSTD along with multiple open source tools and datasets. For more details visit https://shubhanshu.com/phd_thesis/.

‘Bad language’ in the Nordics: profanity and gender in a social media corpus

ABSTRACT This study looks at the relative frequency of ‘bad language’ according to gender in Nordic languages and in English in a 210-million-token corpus of messages by 18,686 Nordic Twitter users. For the Nordic languages, more than 19,000 ‘bad-language’ word forms were compiled on the basis of usage note annotations in major Nordic-language dictionaries. The most frequent terms overall are swear words, and while males use more of these items on average, the gender difference is less pronounced for English words. For potentially offensive words in the Nordic languages, males make more use of traditional profanities associated with the Devil, religion, and blasphemy. Both genders make more use of profanities when tweeting to people of their own gender. The study provides empirical evidence for a small gender-based discrepancy in the use of profanity in social media in the Nordic languages, mirroring results previously found in corpus-based studies of English-language data. The results are interpreted in light of previous findings as evidence for a gendered difference in sensitivity toward the use of language that could potentially be offensive.

Building and Analyzing Panic Disorder Social Media Corpus for Automatic Deep Learning Classification Model

Building and Analyzing Panic Disorder Social Media Corpus for Automatic Deep Learning Classification Model

Deep Learning Based Sentiment Analysis in a Code-Mixed English-Hindi and English-Bengali Social Media Corpus

Sentiment analysis is a circumstantial analysis of text, identifying the social sentiment to better understand the source material. The article addresses sentiment analysis of an English-Hindi and English-Bengali code-mixed textual corpus collected from social media. Code-mixing is an amalgamation of multiple languages, which previously mainly was associated with spoken language. However, social media users also deploy it to communicate in ways that tend to be somewhat casual. The coarse nature of social media text poses challenges for many language processing applications. Here, the focus is on the low predictive nature of traditional machine learners when compared to Deep Learning counterparts, including the contextual language representation model BERT (Bidirectional Encoder Representations from Transformers), on the task of extracting user sentiment from code-mixed texts. Three deep learners (a BiLSTM CNN, a Double BiLSTM and an Attention-based model) attained accuracy 20–60% greater than traditional approaches on code-mixed data, and were for comparison also tested on monolingual English data.