Terrorist Content Research Articles

Mapping interpretations of the law in online content moderation in Germany

Content moderation is a vital condition that online platforms must facilitate, according to the law, to create suitable online environments for their users. By the law, we mean national or European laws that require the removal of content by online platforms, such as EU Regulation 2021/784, which addresses the dissemination of terrorist content online. Content moderation required by these national or European laws, summarised here as ‘the law’, is different from the moderation of pieces of content that is not directly required by law but instead is conducted voluntarily by the platforms. New regulatory requests create an additional layer of complexity of legal grounds for the moderation of content and are relevant to platforms’ daily decisions. The decisions made are either grounded in reasons stemming from different sources of law, such as international or national provisions, or can be based on contractual grounds, such as the platform's Terms of Service and Community Standards. However, how to empirically measure these essential aspects of content moderation remains unclear. Therefore, we ask the following research question: How do online platforms interpret the law when they moderate online content?To understand this complex interplay and empirically test the quality of a platform's content moderation claims, this article develops a methodology that facilitates empirical evidence of the individual decisions taken per piece of content while highlighting the subjective element of content classification by human moderators. We then apply this methodology to a single empirical case, an anonymous medium-sized German platform that provided us access to their content moderation decisions. With more knowledge of how platforms interpret the law, we can better understand the complex nature of content moderation, its regulation and compliance practices, and to what degree legal moderation might differ from moderation due to contractual reasons in dimensions such as the need for context, information, and time.Our results show considerable divergence between the platform's interpretation of the law and ours. We believe that a significant number of platform legal interpretations are incorrect due to divergent interpretations of the law and that platforms are removing legal content that they falsely believe to be illegal (‘overblocking’) while simultaneously not moderating illegal content (‘underblocking’). In conclusion, we provide recommendations for content moderation system design that takes (legal) human content moderation into account and creates new methodological ways to test its quality and effect on speech in online platforms.

Countering online terrorist content: A social regulation approach

AbstractAfter a period of self‐regulation, countries around the world began to implement regulations for the removal of terrorist content from tech platforms. However, much of this regulation has been criticised for a variety of reasons, most prominently for concerns of infringing free speech and creating unfair burdens for smaller platforms. In addition to this, regulation is heavily centred around content moderation, however, fails to consider or address the psychosocial risks it poses to human content moderators. This paper argues that where regulation has been heavily criticised yet continues to inspire similar regulation a new regulatory approach is required. The aim of this paper is to undertake an introductory examination of the use of a social regulation approach in three other industries (environmental protection, consumer protection and occupational health and safety) to learn and investigate new regulatory avenues that could be applied to the development of new regulation that seeks to counter terrorist content on tech platforms and is concerned with the safety of content moderators.

Взаимодействие России и Китая по вопросам противодействия терроризму в киберпространстве

The availability of the Internet, the increasing speed of information dissemination, and the wide reach of audi-ences are moving terrorist organizations into cyberspace. The global nature of the Internet and digitalization requires close international cooperation to counter terrorism in the online space. Over the past 5 years, the problems of cyber terrorism have steadily increased, and both Russia and China remain targets for cyber threats. To counter cyberterrorism, the two countries cooperate multilaterally and bilaterally. Multilateral coop-eration is carried out as part of the UN, SCO, and BRICS, where Russia and China demonstrate similar ap-proaches in developing international legal instruments to combat cybercrime. Bilateral cooperation includes collaboration between national competent authorities, exchange of information between the parties on investi-gating the use of ICTs for criminal purposes, joint response to threats in the information environment, and tech-nological cooperation. Russia and China have achieved some success in countering threats in cyberspace. Nevertheless, it is necessary to expand information interaction between the parties on international platforms, to pursue a policy that prevents terrorist organizations from using the territory of the two countries to commit cybercrimes, and to develop tools for joint control over the dissemination of information with terrorist content.

Content moderation through removal of service: Content delivery networks and extremist websites

AbstractConsiderable attention has been paid by researchers to social media platforms, especially the ‘big companies’, and increasingly also messaging applications, and how effectively they moderate extremist and terrorist content on their services. Much less attention has yet been paid to if and how infrastructure and service providers, further down ‘the tech stack’, deal with extremism and terrorism. Content Delivery Networks (CDN) such as Cloudflare play an underestimated role in moderating the presence of extremist and terrorist content online as it is impossible for these websites to operate without DDoS protection. This is evidenced by the takedown of a wide range websites such as The Daily Stormer, 8chan, a variety of Taliban websites and more recently the organised harassment site Kiwifarms following refusal of service by Cloudflare. However, it is unclear whether there is any formal process of content review conducted by the company when it decides to refuse services. This article aims to first provide an analysis of what extremist and terrorist websites make use of Cloudflare's services as well as other CDNs, and how many of them have been subject to takedown following refusal of service. Following this the article analyses CDNs' terms of service and how current and upcoming internet regulation applies to these CDNs.

Gone in 60 Minutes: Distribution of Terrorist Content and Free Speech in the European Union

ABSTRACT The glorification of terrorist offenses and the dissemination of radical content in digital services pose a serious problem, which requires not only technical but also legal measures. It is therefore not surprising that legislatures, also in response to public expectations, are increasingly eager to implement new laws aimed at the removal of terrorist content. The result, however, is also a number of controversies concerning the proportionality of the measures taken – especially insofar as concerns the “flip-side” aspects of respect for freedom of speech and the creation of preventive censorship mechanisms. An example of such a measure, introduced by EU Regulation 2021/784, is the 1-hour data removal order. These orders, issued by national authorities, make it possible to oblige a digital service provider to delete online content almost immediately. The lack of prior judicial review and the short time to comply with the order are features that, on the one hand, ensure high effectiveness in removing illegal content, but on the other hand increase the risk of abuse of power. The aim of this paper is to discuss the doubts concerning the compliance of 1-hour removal orders with the EU model of fundamental rights protection. Against this background, conclusions will also be drawn from the analysis of similar German and French provisions. As a result, it will be possible to identify areas for improvement in the current legislation, which seems necessary to be taken into account in order for the EU’s actions to support public security objectives without creating the risk of a significant breach of fundamental rights.

The Lazy Legislature: Incorporating and Horizontalising the Charter of Fundamental Rights through Secondary Union Law

Constitutionalising private law – Horizontal effect of fundamental rights –New legislative techniques in the EU in platform governance – References to the Charter of Fundamental Rights in secondary union law – Article 14(4) of the Digital Services Act – Article 5(1) subparagraph 2 of the Terrorist Content Online Regulation – Compatibility of secondary union law references to the Charter with primary union law – Interpretative methodology of secondary union law references

Using Terms and Conditions to apply Fundamental Rights to Content Moderation

AbstractLarge online platforms provide an unprecedented means for exercising freedom of expression online and wield enormous power over public participation in the online democratic space. However, it is increasingly clear that their systems, where (automated) content moderation decisions are taken based on a platformʼs terms and conditions (T&Cs), are fundamentally broken. Content moderation systems have been said to undermine freedom of expression, especially where important public interest speech ends up suppressed, such as speech by minority and marginalized groups. Indeed, these content moderation systems have been criticized for their overly vague rules of operation, inconsistent enforcement, and an overdependence on automation. Therefore, in order to better protect freedom of expression online, international human rights bodies and civil society organizations have argued that platforms “should incorporate directly” principles of fundamental rights law into their T&Cs. Under EU law, and apart from a rule in the Terrorist Content Regulation, platforms had until recently no explicit obligation to incorporate fundamental rights into their T&Cs. However, an important provision in the Digital Services Act (DSA) will change this. Crucially, Article 14 DSA lays down new rules on how platforms can enforce their T&Cs, including that platforms must have “due regard” to the “fundamental rights” of users under the EU Charter of Fundamental Rights. In this article, we critically examine the topic of enforceability of fundamental rights via T&Cs through the prism of Article 14 DSA. We ask whether this provision requires platforms to apply EU fundamental rights law and to what extent this may curb the power of Big Tech over online speech. We conclude that Article 14 will make it possible, in principle, to establish the indirect horizontal effect of fundamental rights in the relationship between online platforms and their users. But in order for the application and enforcement of T&Cs to take due regard of fundamental rights, Article 14 must be operationalized within the framework of the international and European fundamental rights standards. If this is possible Article 14 may fulfil its revolutionary potential.

TamTam: The Online Drums of Hate

Recognizing the correlation between the growing spread of violent extremist and terrorist content on the internet and online platforms and the significant increase in attacks inspired by these posts prompted governments, security agencies, and private companies to launch various countermeasures to reduce the spread and impact of such material. Struggling to adjust to these countermeasures and maintain their online presence, terrorists and violent extremists have migrated from mainstream online platforms to alternative online channels like 4chan, 8chan, Telegram, TikTok, and most recently to the new platform TamTam. This paper describes the migration to and within social media generally, with a specific focus on the newly discovered migration to the platform TamTam and presents the results of our extensive content analysis of hundreds of posts and shares containing terrorist and violent extremist content on this platform. These findings are discussed in terms of ethical and practical implications.

Introduction to Special Issue: The Practicalities and Complexities of (Regulating) Online Terrorist Content Moderation

Introduction to Special Issue: The Practicalities and Complexities of (Regulating) Online Terrorist Content Moderation

Developing a Responsive Regulatory Approach to Online Terrorist Content on Tech Platforms

Tech platforms have been through several regulatory phases concerning the countering of terrorist content on their platforms: from a lack of regulation, to the government demands of self-regulation, to the present day where regulation has been implemented across the world. Much of this regulation, however, has been heavily criticised. After consideration of these criticisms, and research into regulatory approaches in a range of other industries, this article proposes a responsive regulatory approach to countering online terrorist content. Responsive regulation argues that regulation should be responsive to specific industry structure because different structures are conducive to different degrees and forms of regulation. This article categorizes tech platforms based on different compliance issues: awareness and expertise; capacity; and willingness. It then proposes four regulatory tracks in response, in order to try to mitigate the occurrence of these compliance issues.

Negotiating Fundamental Rights: Civil Society and the EU Regulation on Addressing the Dissemination of Terrorist Content Online

During the negotiations of the EU Regulation on addressing the dissemination of terrorist content online, digital and human rights activists raised concerns regarding the safeguarding of fundamental rights. These actors were partially successful in influencing aspects of the final text of the Regulation, thus, raising the question of how such groups have managed to influence traditionally “securitized” counterterrorism policy. This study compares the discourses employed by the different EU institutions and digital and human rights advocates during the negotiation period. Approaching this through the lens of politicization, this study demonstrates that whilst securitizing discourses persist to a certain extent, some aspects of EU counterterrorism law are open to contestation.

SoK: Content Moderation for End-to-End Encryption

Popular messaging applications now enable end-to-end-encryption (E2EE) by default, and E2EE data storage is becoming common. These important advances for security and privacy create new content moderation challenges for online services, because services can no longer directly access plaintext content. While ongoing public policy debates about E2EE and content moderation in the United States and European Union emphasize child sexual abuse material and misinformation in messaging and storage, we identify and synthesize a wealth of scholarship that goes far beyond those topics. We bridge literature that is diverse in both content moderation subject matter, such as malware, spam, hate speech, terrorist content, and enterprise policy compliance, as well as intended deployments, including not only privacy-preserving content moderation for messaging, email, and cloud storage, but also private introspection of encrypted web traffic by middleboxes. In this work, we systematize the study of content moderation in E2EE settings. We set out a process pipeline for content moderation, drawing on a broad interdisciplinary literature that is not specific to E2EE. We examine cryptography and policy design choices at all stages of this pipeline, and we suggest areas of future research to fill gaps in literature and better understand possible paths forward.

The Digital Services Act (DSA): A New Era for Online Harms and Intermediary Liability

The EU’s Digital Services Act (DSA) marks the biggest shake up to the rules for online intermediary liability in twenty years. The DSA is accompanied by flanking instruments regulating terrorist content, child sexual abuse material (CSAM) and political ads which, together, will create an entirely new framework for the regulation of online harms in the EU. These new and wide-ranging obligations attempt to reconcile the damage caused by unregulated user-generated content, fundamental rights to freedom of information and the practical limitations of moderating content at scale. The DSA is likely to shape the global approach to content regulation in this emerging area of law. Digital Services Act, DSA, Online Harm, Intermediary Liability, Intermediary Services, Hosting Services, Very Large Online Search Engines, Online Platforms, Very Large Online Platforms

Legal and technical trade-offs in the content moderation of terrorist live-streaming

Abstract Moderating terrorist live-streaming presents legal and technical trade-offs. The immediacy of live-streamed content requires a timely assessment of the content’s compliance with community guidelines and an even more expeditious action of restriction or removal in case of portrayal of illegal content. Social media companies have heavily relied on technology to moderate content. The most frequently used tools to screen and filter content and ensure compliance with community standards and regulations include hashing technology, video-fingerprinting, natural language processing and metadata analysis. Terrorist content online presents nuances that make the task of removing it particularly challenging especially when companies must rely on machine learning content moderation. We identify three trade-offs in regulating content moderation of live-streamed content: technological neutrality v specificity, explainability v adversarial machine learning, and accuracy v time-efficiency. For each trade-off, we provide an analysis of the considerations lawmakers and regulators must take into account when balancing competing interests.

Open Archive of Hate: Terrorism and Violent Extremism on Internet Archive

The recognized correlation between the growing spread of violent extremist and terrorist content on the Internet and online platforms and the significant increase of attacks inspired by these postings caused governments and security agencies to launch various countermeasures. These measures included removal of terrorist and violent extremist online content (or “deplatforming”), suspension of their social media accounts, hacking websites and pressuring social media companies to remove terrorist propaganda. Struggling to adjust to these countermeasures and sustain their online presence, terrorist and violent extremists moved from mainstream online platforms to alternative online channels including anonymous sharing portals and cloud services. The most popular archiving service used by various extremist and terrorist group is the Internet Archive. While the few studies on hate speech and violent contents on Internet Archive focused only on a certain type of content or a specific group, this study attempts to reveal a wider range of contents, groups and organizations who use this archive for sharing and promoting such contents. To examine the use of Internet Archive by various groups and organizations involved in violence, terrorism, hate speech, racism and neo-Nazism we applied several stages of data collection and analysis. The findings highlight an alarming volume of terrorist, extremist, and racist material on the Internet Archive. These findings are discussed in terms of ethical and practical implications.

ISIL Terrorists and the use of social media platforms: Are offensive and proactive cyber-attacks the solution to the online presence of ISIL?

Private social media platforms are entrusted today with the task of limiting the presence of terrorist groups, such as the Islamic State, on the online scene. Through mechanisms, such as a ‘notice-and-takedown’ or a removal order, these platforms are made aware of online terrorist content and consequently remove it from their platforms. However, the responsibility to protect citizens against terrorist threats lies with states and not with private companies. To reclaim this responsibility, states could perpetrate offensive and proactive cyber-attacks on devices of terrorists located in Belgium to disable their access to information to perpetrate attacks or diminish their online presence. This article discusses the legal frameworks of these mechanisms, their advantages and disadvantages and whether the Belgian state can perform such cyber-attacks or whether it would be desirable for the Belgian authorities to receive this competence. Last, several recommendations are presented to adapt the current Belgian framework to allow the perpetration of such cyber-attacks.

Unique solutions of the French Republic in the fight against terrorism and radicalisation

Experience and solutions of the French Republic, where anti-terrorist structures are constantly developed, should be a signpost for other countries. Creation of unique deradicalization programs and conducting a policy of responding to cases of extremism are actions that can be implemented in Poland. In addition, the use of social media to combat terrorist content and to provide information about threats shows the innovative use of the Internet. It is also worth noting the involvement of French municipal guards and medical services in the process of responding to terrorism, as well as highlighting the role of NGOs working to support victims. Decades of France’s struggle with different types of terrorism have meant that methods that would be effective in countering radicalisation and threats have been improved and reviewed over the years. In the author’s opinion, it would be advisable to take advantage of the French experience and implement at least some of the solutions in Poland.

Dissemination of terrorist content online and service provider liability: an overview of the EU legal framework

Nowadays, due to the global reach of technology, the Internet is the most commonly used medium of information. The purpose of this article is to illustrate what the European Union (EU) has done so far regarding the fight against dissemination of terrorist content through digital platforms. Both hard law and soft measures were adopted to set rules and obligations the service providers must comply with in order to prevent the dissemination of terrorist content through their services. The conclusion discusses the need to find a balance between the protection of fundamental rights and public security and includes some suggestions to improve existing rules. The methodology used includes documentary analysis and bibliographic research.

A Dual Challenge to Human Rights Law: Online Terrorist Content and Governmental Orders to Remove it

A Dual Challenge to Human Rights Law: Online Terrorist Content and Governmental Orders to Remove it

Co-Producing Security: Platform Content Moderation and European Security Integration.

The European Union (EU) seeks to play a leading role in steering the private work of online content moderation, as demonstrated by numerous policy and legislative initiatives in the domain. Two initiatives, in particular, are shaping terrorist content moderation: the creation of a EU Internet Referral Unit and the adoption of a Regulation on preventing the online dissemination of terrorist content (TERREG). This article analyses these initiatives and their practical effects. In particular, it unpacks the legal and technological mechanisms at the core of EU regulation in the realm of online terrorist content moderation, and how they co‐produce security decisions across public and private spheres. Based on interviews, fieldwork observations and document analysis, we show how processes of referral and removal, and processes of flagging and filtering are key to EU‐directed content moderation. In conclusion, we reflect on content moderation as a novel form of European security integration.