Computational Social Science Research Research Articles

Bootstrapping public entities. Domain-specific NER for public speakers

ABSTRACT Named Entity Recognition (NER) is a supervised machine learning task that finds various applications in automated content analysis, as the identification of entities is vital for understanding public discourse. However, sometimes the standard NER labels are not specific enough for a given domain. We introduce Public Entity Recognition (PER). PER is a domain-specific version of NER, that is trained for five entity types that are common to public discourse: politicians, parties, authorities, media, and journalists. PER can be used for pre-processing documents, in a pipeline with other classifiers or directly for analyzing information in texts. The taxonomy for PER is taken from the database of (German) public speakers and aims at low-threshold integration into computational social science research. We experiment with different training settings, involving weakly supervised training and training on manually annotated data. We evaluate multilingual transformer models of different sizes against rule-based entity matching and find that the models do not only outperform the baseline but also reach competitive absolute scores of around .8 and higher in F1. We further test for generalization and domain adaptation. We show that with only around 100–150 additional sentences, the model can be adapted to new languages.

No recognised ethical standards, no broad consent: navigating the quandary in computational social science research

Recital 33 GDPR has often been interpreted as referring to ‘broad consent’. This version of informed consent was intended to allow data subjects to provide their consent for certain areas of research, or parts of research projects, conditional to the research being in line with ‘recognised ethical standards’. In this article, we argue that broad consent is applicable in the emerging field of Computational Social Science (CSS), which lies at the intersection of data science and social science. However, the lack of recognised ethical standards specific to CSS poses a practical barrier to the use of broad consent in this field and other fields that lack recognised ethical standards. Upon examining existing research ethics standards in social science and data science, we argue that they are insufficient for CSS. We further contend that the fragmentation of European Union (EU) law and research ethics sources makes it challenging to establish universally recognised ethical standards for scientific research. As a result, CSS researchers and other researchers in emerging fields that lack recognised ethical standards are left without sufficient guidance on the use of broad consent as provided for in the GDPR. We conclude that responsible EU bodies should provide additional guidance to facilitate the use of broad consent in CSS research.

Quali-quanti visual methods and political bots

Computational social science research on automated social media accounts, colloquially dubbed “bots”, has tended to rely on binary verification methods to detect bot operations on social media. Typically focused on textual data from Twitter (now rebranded as "X"), these inference-based methods are prone to finding false positives and failing to understand the subtler ways in which bots operate over time, through visual content and in particular contexts. This research brings methodological contributions to such studies, focusing on what it calls “bolsobots” in Brazilian social media. Named after former Brazilian President Jair Bolsonaro, the bolsobots refer to the extensive and skilful usage of partial or fully automated accounts by marketing teams, hackers, activists or campaign supporters. These accounts leverage online political culture to sway public opinion for or against public policies, opposition figures, or Bolsonaro himself. Drawing on empirical case studies, this paper implements quali-quanti visual methods to operationalise specific techniques for interpreting bot-associated image collections and textual content across Instagram, TikTok and Twitter/X. To unveil the modus operandi of bolsobots, we map the networks of users they follow (“following networks”), explore the visual-textual content they post, and observe the strategies they deploy to adapt to platform content moderation. Such analyses tackle methodological challenges inherent in bot studies by employing three key strategies: 1) designing context-sensitive queries and curating datasets with platforms’ interfaces and search engines to mitigate the limitations of bot scoring detectors, 2) engaging qualitatively with data visualisations to understand the vernaculars of bots, and 3) adopting a non-binary analysis framework that contextualises bots within their socio-technical environments. By acknowledging the intricate interplay between bots, user and platform cultures, this paper contributes to method innovation on bot studies and emerging quali-quanti visual methods literature.

Building an ICCN Multimodal Classifier of Aggressive Political Debate Style: Towards a Computational Understanding of Candidate Performance Over Time

ABSTRACT Understanding the implications of aggressive political debate style amid corrosive modes of campaign politics requires fine-grained analyses of political performance, attending to multiple communication modalities. Politicians’ facial expressions, emotional tone, and speech content can all independently convey aggression and dominance, and often work in combination for purposes of emphasis. Yet micro-coding individual visual, tonal, and verbal features across more than a handful of debate segments becomes extremely labor intensive, hampering research, especially historical, longitudinal, and cross-cultural work. To address this limitation, we develop a novel multimodal classifier using an Interaction Canonical Correlation Network (ICCN) that incorporates video and audio features with speech coding of candidate debate performance, trained on a 20% sample of 10-second segments from each of the first televised U.S presidential debates between 1980 and 2020. In the analysis, we demonstrate this classifier can accurately detect aggression by political candidates in U.S. debates. We sharpen its performance by distinguishing between debate eras characterized by lower and higher levels of aggression and validate the approach by comparing the performance of unimodal with multimodal classification. This classifier opens new avenues for computational social science research, including explaining candidate behavior within debates at a larger scale and across different eras.

Just Another Day on Twitter: A Complete 24 Hours of Twitter Data

At the end of October 2022, Elon Musk concluded his acquisition of Twitter. In the weeks and months before that, several questions were publicly discussed that were not only of interest to the platform's future buyers, but also of high relevance to the Computational Social Science research community. For example, how many active users does the platform have? What percentage of accounts on the site are bots? And, what are the dominating topics and sub-topical spheres on the platform? In a globally coordinated effort of 80 scholars to shed light on these questions, and to offer a dataset that will equip other researchers to do the same, we have collected all 375 million tweets published within a 24-hour time period starting on September 21, 2022. To the best of our knowledge, this is the first complete 24-hour Twitter dataset that is available for the research community. With it, the present work aims to accomplish two goals. First, we seek to answer the aforementioned questions and provide descriptive metrics about Twitter that can serve as references for other researchers. Second, we create a baseline dataset for future research that can be used to study the potential impact of the platform's ownership change.

Building a Model for Integrative Computational Social Science Research

Universities are increasingly interested in providing courses that equip students with data science skills and engage experiential learning, particularly in the social sciences. However, these courses can be costly to implement and time-consuming for instructors to develop. This article describes an integrative learning model for teaching computational social science skills to undergraduate students. There are three elements to the model: content delivery through collaborative learning, skill development in an applied lab setting, and student mentorship. I apply this model to an experiential course where undergraduate students learn to create and conduct a national public opinion poll. The model addresses the need for university classes that equip undergraduate students with computational social science skills, and provides a pathway for training student researchers as teaching assistants for future courses. This model builds the research capacity of faculty, graduate assistants, and undergraduates, invests in data science by providing an infrastructure for the collection of large scale data over time, and integrates experiences inside and outside of the classroom. This model is replicable at other institutions and will be of benefit to programs seeking to implement best practices and maximize learning effectiveness through research integration.

Twitter spam and false accounts prevalence, detection, and characterization: A survey

The issue of quantifying and characterizing various forms of social media manipulation and abuse has been at the forefront of the computational social science research community for over a decade. In this paper, I provide a (non-comprehensive) survey of research efforts aimed at estimating the prevalence of spam and false accounts on Twitter, as well as characterizing their use, activity, and behavior. I propose a taxonomy of spam and false accounts, enumerating known techniques used to create and detect them. Then, I summarize studies estimating the prevalence of spam and false accounts on Twitter. Finally, I report on research that illustrates how spam and false accounts are used for scams and frauds, stock market manipulation, political disinformation and deception, conspiracy amplification, coordinated influence, public health misinformation campaigns, radical propaganda and recruitment, and more. I will conclude with a set of recommendations aimed at charting the path forward to combat these problems.

Paths Study on Knowledge Convergence and Development in Computational Social Science: Data Metric Analysis Based on Web of Science

Computational social science, as an emerging interdisciplinary discipline, is a field ushered in by long‐term development of traditional social science. It is committed to supplying data thinking, resources, and analytics to study human social behavior and social operation laws to accurately grasp and judge the developing path of the discipline, which is of great significance to promote the innovation and development of social sciences. This study is to conduct a systematic quantitative analysis from a bibliometric perspective, aiming to provide a reference for scholars to explore the paths and changing rules in the field. We use the relevant literature in Web of Science as the dataset. After eliminating journal calls and irrelevant literature, R language and SciMAT tools are used to visualize and analyze the number of articles, keyword clustering, keyword cooccurrence network, and theme evolution, so as to summarize and sort out the paths of computational social science research. The study found that the annual volume of publications has been gradually increasing and will probably remain active in the next few years with high productivity. Subject themes in different periods are diversified, and the evolutionary relationship is found complex as well. Besides, as a cross discipline, scientific knowledge from different fields cross collides and couples with each other in the big data environment, changing the traditional concept of computational social science and forming a new development path. Recently, the emergence of “big data+” has promoted the rise of new subject areas, making the development of new disciplines a reality.

Suicide Risk and Protective Factors in Online Support Forum Posts: Annotation Scheme Development and Validation Study.

BackgroundOnline communities provide support for individuals looking for help with suicidal ideation and crisis. As community data are increasingly used to devise machine learning models to infer who might be at risk, there have been limited efforts to identify both risk and protective factors in web-based posts. These annotations can enrich and augment computational assessment approaches to identify appropriate intervention points, which are useful to public health professionals and suicide prevention researchers.ObjectiveThis qualitative study aims to develop a valid and reliable annotation scheme for evaluating risk and protective factors for suicidal ideation in posts in suicide crisis forums.MethodsWe designed a valid, reliable, and clinically grounded process for identifying risk and protective markers in social media data. This scheme draws on prior work on construct validity and the social sciences of measurement. We then applied the scheme to annotate 200 posts from r/SuicideWatch—a Reddit community focused on suicide crisis.ResultsWe documented our results on producing an annotation scheme that is consistent with leading public health information coding schemes for suicide and advances attention to protective factors. Our study showed high internal validity, and we have presented results that indicate that our approach is consistent with findings from prior work.ConclusionsOur work formalizes a framework that incorporates construct validity into the development of annotation schemes for suicide risk on social media. This study furthers the understanding of risk and protective factors expressed in social media data. This may help public health programming to prevent suicide and computational social science research and investigations that rely on the quality of labels for downstream machine learning tasks.

#Election2020: the first public Twitter dataset on the 2020 US Presidential election

Credible evidence-based political discourse is a critical pillar of democracy and is at the core of guaranteeing free and fair elections. The study of online chatter is paramount, especially in the wake of important voting events like the recent November 3, 2020 U.S. Presidential election and the inauguration on January 21, 2021. Limited access to social media data is often the primary obstacle that limits our abilities to study and understand online political discourse. To mitigate this impediment and empower the Computational Social Science research community, we are publicly releasing a massive-scale, longitudinal dataset of U.S. politics- and election-related tweets. This multilingual dataset encompasses over 1.2 billion tweets and tracks all salient U.S. political trends, actors, and events from 2019 to the time of this writing. It predates and spans the entire period of the Republican and Democratic primaries, with real-time tracking of all presidential contenders on both sides of the aisle. The dataset also focuses on presidential and vice-presidential candidates, the presidential elections and the transition from the Trump administration to the Biden administration. Our dataset release is curated, documented, and will continue to track relevant events. We hope that the academic community, computational journalists, and research practitioners alike will all take advantage of our dataset to study relevant scientific and social issues, including problems like misinformation, information manipulation, conspiracies, and the distortion of online political discourse that has been prevalent in the context of recent election events in the United States. Our dataset is available at: https://github.com/echen102/us-pres-elections-2020.

Mobile Phone Indicators and Their Relation to the Socioeconomic Organisation of Cities

Thanks to the use of geolocated big data in computational social science research, the spatial and temporal heterogeneity of human activities is increasingly being revealed. Paired with smaller and more traditional data, this opens new ways of understanding how people act and move, and how these movements crystallise into the structural patterns observed by censuses. In this article we explore the convergence between mobile phone data and more traditional socioeconomic data from the national census in French cities. We extract mobile phone indicators from six months worth of Call Detail Records (CDR) data, while census and administrative data are used to characterize the socioeconomic organisation of French cities. We address various definitions of cities and investigate how they impact the statistical relationships between mobile phone indicators, such as the number of calls or the entropy of visited cell towers, and measures of economic organisation based on census data, such as the level of deprivation, inequality and segregation. Our findings show that some mobile phone indicators relate significantly with different socioeconomic organisation of cities. However, we show that relations are sensitive to the way cities are defined and delineated. In several cases, changing the city delineation rule can change the significance and even the sign of the correlation. In general, cities delineated in a restricted way (central cores only) exhibit traces of human activity which are less related to their socioeconomic organisation than cities delineated as metropolitan areas and dispersed urban regions.

Assessing the Accuracy of Four Popular Face Recognition Tools for Inferring Gender, Age, and Race

In this research, we evaluate four widely used face detection tools, which are Face++, IBM Bluemix Visual Recognition, AWS Rekognition, and Microsoft Azure Face API, using multiple datasets to determine their accuracy in inferring user attributes, including gender, race, and age. Results show that the tools are generally proficient at determining gender, with accuracy rates greater than 90%, except for IBM Bluemix. Concerning race, only one of the four tools provides this capability, Face++, with an accuracy rate of greater than 90%, although the evaluation was performed on a high-quality dataset. Inferring age appears to be a challenging problem, as all four tools performed poorly. The findings of our quantitative evaluation are helpful for future computational social science research using these tools, as their accuracy needs to be taken into account when applied to classifying individuals on social media and other contexts. Triangulation and manual verification are suggested for researchers employing these tools.

Social science, today Bit by Bit: Social Research in the Digital Age Matthew J. Salganik Princeton University Press, 2017. 445 pp.

A breezy, personal guide provides a road map to solid computational social science research

Education policy research in the big data era: Methodological frontiers, misconceptions, and challenges

Despite abundant data and increasing data availability brought by technological advances, there has been very limited education policy studies that have capitalized on big data—characterized by large volume, wide variety, and high velocity. Drawing on the recent progress of using big data in public policy and computational social science research, this commentary discusses how to approach big data and how big data can be used in education policy research. First, I introduce big data that is potentially relevant to education policy research. I then present methodological frontiers by examining the assumptions, key concepts, merits, and caveats of three commonly used analytical approaches to mining massive amounts of text data: topic models, network text analysis, and sentiment analysis. Next, to ensure the veracity of using big data in education policy research, I debunk three methodological misconceptions. This commentary concludes with a discussion on developing interdisciplinary research capacity and addressing the privacy concerns and ethical conundrums as we explore a research agenda of using big data in education policy.

Citizen Science Practices for Computational Social Science Research: The Conceptualization of Pop-Up Experiments

Under the name of Citizen Science, many innovative practices in which volunteers partner up with scientists to pose and answer real-world questions are growing rapidly worldwide. Citizen Science can furnish ready-made solutions with citizens playing an active role. However, this framework is still far from being well established as a standard tool for computational social science research. Here, we present our experience in bridging gap between computational social science and the philosophy underlying Citizen Science, which in our case has taken the form of what we call ``pop-up experiments. These are non-permanent, highly participatory collective experiments which blend features developed by big data methodologies and behavioural experimental protocols with the ideals of Citizen Science. The main issues to take into account whenever planning experiments of this type are classified, discussed and grouped into three categories: infrastructure, public engagement, and the knowledge return for citizens. We explain the solutions we have implemented, providing practical examples grounded in our own experience in an urban context (Barcelona, Spain). Our aim here is that this work will serve as a guideline for groups willing to adopt and expand such \emph{in-vivo} practices and we hope it opens up the debate regarding the possibilities (and also the limitations) that the Citizen Science framework can offer the study of social phenomena.

Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora.

A word's sentiment depends on the domain in which it is used. Computational social science research thus requires sentiment lexicons that are specific to the domains being studied. We combine domain-specific word embeddings with a label propagation framework to induce accurate domain-specific sentiment lexicons using small sets of seed words. We show that our approach achieves state-of-the-art performance on inducing sentiment lexicons from domain-specific corpora and that our purely corpus-based approach outperforms methods that rely on hand-curated resources (e.g., WordNet). Using our framework, we induce and release historical sentiment lexicons for 150 years of English and community-specific sentiment lexicons for 250 online communities from the social media forum Reddit. The historical lexicons we induce show that more than 5% of sentiment-bearing (non-neutral) English words completely switched polarity during the last 150 years, and the community-specific lexicons highlight how sentiment varies drastically between different communities.

Dealing with Data: Governments Records

In his Perspective “Ensuring the data-rich future of the social sciences” (special section on Dealing with Data, 11 February, p. [719][1]), G. King discusses the potential to transform the future of social science using existing “huge quantities of digital information about people.” As King states, the business sector has proven that powerful informatics applied to data about people can support better decision-making and foster innovation. The same could be done to tackle difficult social problems once we figure out how to assemble and share large data sets while protecting privacy. King suggests building data archives to facilitate reuse of data. Large data archives already exist on survey data ([ 1 ][2]–[ 5 ][3]), but little attention has been given to government administrative data, which is a key source of information about our society. From the day we are born until we die, most of our activities leave traces in various government databases. Indeed, these government information systems continuously generate data about all aspects of our society, much like the satellites we use to monitor our physical surroundings. Unfortunately, most administrative data are left to languish in legacy databases after their original use. We must invest in efforts to build a systematic pipeline to extract data from these systems for secondary analysis. A well-integrated federated data system of administrative databases updated on an ongoing basis could hold a collective representation of our society. Designed, built, and properly managed by experts under appropriate protocols and oversight, such a data infrastructure could transform the social sciences and still protect individual privacy, just as well-maintained satellite data have transformed astronomy. A secure data infrastructure is critical for government decision support systems, transparency and accountability in government, and ultimately computational social science research. The Census Bureau ([ 5 ][3]) and others ([ 6 ][4], [ 7 ][5]) have already demonstrated the enormous potential of federated administrative data systems. 1. [↵][6] The Centers for Disease Control and Prevention (CDC) National Center for Health Statistics Research Data Center ([www.cdc.gov/rdc/][7]). 2. The Dataverse Network Project ( ). 3. NORC Data Enclave ([www.norc.uchicago.edu/DataEnclave/default.htm][8]). 4. Odum Institute Dataverse Network Project ( ). 5. [↵][9] U.S. Census Bureau Longitudinal Employer Household Dynamics (LEHD) ( ). 6. [↵][10] U.S. Environmental Protection Agency, Central Data Exchange ([www.epa.gov/cdx/index.htm][11]). 7. [↵][12] 1. H. C. Kum, 2. D. F. Duncan, 3. C. J. Stewart , Gov. Inform. Q. 26, 295 (2009). [OpenUrl][13][CrossRef][14] [1]: /lookup/doi/10.1126/science.1197872 [2]: #ref-1 [3]: #ref-5 [4]: #ref-6 [5]: #ref-7 [6]: #xref-ref-1-1 View reference 1 in text [7]: http://www.cdc.gov/rdc/ [8]: http://www.norc.uchicago.edu/DataEnclave/default.htm [9]: #xref-ref-5-1 View reference 5 in text [10]: #xref-ref-6-1 View reference 6 in text [11]: http://www.epa.gov/cdx/index.htm [12]: #xref-ref-7-1 View reference 7 in text [13]: {openurl}?query=rft.jtitle%253DGov.%2BInform.%2BQ.%26rft.volume%253D26%26rft.spage%253D295%26rft_id%253Dinfo%253Adoi%252F10.1016%252Fj.giq.2008.12.009%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [14]: /lookup/external-ref?access_num=10.1016/j.giq.2008.12.009&link_type=DOI

Collecting legacy corpora from social science research for text mining evaluation

In this poster we describe a pilot study of searching social science literature for legacy corpora to evaluate text mining algorithms. The new emerging field of computational social science demands large amount of social science data to train and evaluate computational models. We argue that the legacy corpora that were annotated by social science researchers through traditional Qualitative Data Analysis (QDA) are ideal data sets to evaluate text mining methods, such as text categorization and clustering. As a pilot study, we searched articles that involve content analysis and discourse analysis in leading communication journals, and then contacted the authors regarding the availability of the annotated texts. Regretfully, nearly all of the corpora that we found were not adequately maintained, and many were no longer available, even though they were less than ten years old. This situation calls for more effort to better maintain and use legacy social science data for future computational social science research purpose.