Open Source Intelligence Research Articles

Expertise hubs and the credibility challenge for open-source intelligence: insights from usage patterns of a web-controlled radio receiver and related Twitter traffic in the Ukraine war

Expertise hubs and the credibility challenge for open-source intelligence: insights from usage patterns of a web-controlled radio receiver and related Twitter traffic in the Ukraine war

Research by the method of an extended systematical literature review E-SLR the problem of ensuring the security of personal data when using OSINT

Objective. The amount of personal data in open sources increases, which makes it possible for third parties to access it using open source intelligence (OSINT) methods, which can be used for malicious purposes. The aim of the work is to identify threats and existing methods and means of ensuring the security of a user's personal data and his reputation when using OSINT by intruders, as well as to identify the main problems in protecting user PD taking into account OSINT. Method. The study uses an extended method of systematic literature review (e-SLR), which is a systematic literature review (SLR) supplemented by responses from ChatGPT, GigaCHAT, YndexGPT neural networks. Result. 41 sources were received for the analysis of the problem, on the basis of which threats to personal data were identified: violation of the confidentiality of personal data and the operation of information systems, targeted attacks using social engineering, password disclosure, espionage; protection tools: data processing before publication, anonymization and depersonalization, limitation of personal data, selection of sites, protection using OSINT, creation of complex passwords, use of protection tools, organizational measures; problems in the development of protection tools: working with big data, unreliability of information and sources, labor-intensiveness of data analysis, technical limitations, bias, ethical and legal aspects. Conclusion. The results were used to develop models for protecting personal data in open sources, methods and means for detecting and preventing violations of their security.

Development of Digital Forensic Framework for Anti-Forensic and Profiling Using Open Source Intelligence in Cyber Crime Investigation

Abstract. Cybercrime is a crime that increases every year. The development of cyber crime occurs by utilizing mobile devices such as smartphones. So it is necessary to have a scientific discipline that studies and handles cybercrime activities. Digital forensics is one of the disciplines that can be utilized in dealing with cyber crimes. One branch of digital forensic science is mobile forensics which studies forensic processes on mobile devices. However, in its development, cybercriminals also apply various techniques used to thwart the forensic investigation process. The technique used is called anti-forensics. Purpose: It is necessary to have a process or framework that can be used as a reference in handling cybercrime cases in the forensic process. This research will modify the digital forensic investigation process. The stages of digital forensic investigations carried out consist of preparation, preservation, acquisition, examination, analysis, reporting, and presentation stages. The addition of the use of Open Source Intelligence (OSINT) and toolset centralization at the analysis stage is carried out to handle anti-forensics and add information from digital evidence that has been obtained in the previous stage. Methods/Study design/approach: This research will modify the digital forensic investigation process. The stages of digital forensic investigations carried out consist of preparation, preservation, acquisition, examination, analysis, reporting, and presentation stages. The addition of the use of Open Source Intelligence (OSINT) and toolset centralization at the analysis stage is carried out to handle anti-forensics and add information from digital evidence that has been obtained in the previous stage. By testing the scenario data, the results are obtained in the form of processing additional information from the files obtained and information related to user names. Result/Findings: The result is a digital forensic phase which concern on anti-forensic identification on media files and utilizing OSINT to perform crime suspect profiling based on the evidence collected in digital forensic investigation phase. Novelty/Originality/Value: Found 3 new types of findings in the form of string data, one of which is a link, and 7 new types in the form of usernames which were not found in the use of digital forensic tools. From a total of 408 initial data and new findings with a total of 10 findings, the percentage of findings increased by 2.45%.

COMPUTER VISION METHODS FOR CONDUCTING OSINT INVESTIGATIONS

This paper researches the application of computer vision techniques in concern with OSINT (Open Source Intelligence) investigations. It explores how state-ofthe- art algorithms and models in computer vision can be used to automate and enhance the process of gathering, analyzing, and interpreting visual data from open sources. The research focuses on the critical steps of image scraping, data preprocessing, and embedding generation using advanced deep learning models such as CLIP. Additionally, the study examines the challenges of managing large-scale visual data and implementing efficient search mechanisms through vector databases like Faiss and Weaviate. By applying these technologies, the paper illustrates how investigators can improve the accuracy and efficiency of imagebased searches, which are later used for uncovering hidden connections and verifying information in OSINT investigations. The findings contribute to the growing field of computer vision and intelligence gathering, offering practical recommendations for enhancing investigative processes through the integration of computer vision methodologies.

Balancing National Security and Privacy: Examining the Use of Commercially Available Information in OSINT Practices

Open source intelligence (OSINT) researchers utilize specialized tools to access vast amounts of data from multiple sources simultaneously. These tools, equipped with (paid) modules, allow users to tap into aggregated data sets containing commercially available information, such as location data from mobile phone users. The utilization of commercially available information from OSINT tools by intelligence and security services impacts fundamental rights and freedoms; more specifically, the right to personal data protection. Drawing from prior experience working on this topic within a Dutch oversight committee on the intelligence and security services and international developments in OSINT practice, insights are provided on this new OSINT practice and the responses of oversight authorities. Rather than advocating for a categorical ban, a more refined approach to process commercially available information from OSINT tools is suggested. Building on the work of a Dutch oversight authority and the work of the U.S. Office of the Director of National Intelligence, four recommendations are provided to intelligence and security services to responsibly handle commercially available information in OSINT practices.

Open-source intelligence and great-power competition under mediatization

Open-source intelligence and great-power competition under mediatization

A Common Effort: New Divisions of Labor Between Journalism and OSINT Communities on Digital Platforms

This article explores the interactions between journalistic actors and emerging open-source intelligence and investigation (OSINT) communities. It employs qualitative content analysis of discourse from two OSINT communities surrounding three events following the Russian invasion of Ukraine in 2022, which received substantial coverage in news media. OSINT practices are rapidly becoming a mainstay of the contemporary political process by allowing ordinary citizens to verify information shared through digital platforms, which is traditionally the societal task assigned to journalism. In doing so, they provide a timely factual baseline for opinion formation and political decision-making. This research explores the role constellations resulting from this shift in verification duties from journalistic actors to amateur online communities on digital platforms and maps the fundamental dynamics involved in OSINT. We analyze how information is received and processed in OSINT communities, how digital platforms facilitate the fact-checking process, and how journalism and OSINT interact. Based on our findings, we develop a theoretical framework that distinguishes between the input, throughput, and output phases of OSINT. Our model contributes to a baseline understanding of the crucial and novel partnership between citizens and journalists on digital platforms.

Use of Open-Source Epidemic Intelligence for Infectious Disease Outbreaks, Ukraine, 2022.

Formal infectious disease surveillance in Ukraine has been disrupted by Russia's 2022 invasion, leading to challenges with tracking and containing epidemics. To analyze the effects of the war on infectious disease epidemiology, we used open-source data from EPIWATCH, an artificial intelligence early-warning system. We analyzed patterns of infectious diseases and syndromes before (November 1, 2021-February 23, 2022) and during (February 24-July 31, 2022) the conflict. We compared case numbers for the most frequently reported diseases with numbers from formal sources and found increases in overall infectious disease reports and in case numbers of cholera, botulism, tuberculosis, HIV/AIDS, rabies, and salmonellosis during compared with before the invasion. During the conflict, although open-source intelligence captured case numbers for epidemics, such data (except for diphtheria) were unavailable/underestimated by formal surveillance. In the absence of formal surveillance during military conflicts, open-source data provide epidemic intelligence useful for infectious disease control.

Use and Abuse of Personal Information, Part I: Design of a Scalable OSINT Collection Engine

In most open-source intelligence (OSINT) research efforts, the collection of information is performed in an entirely passive manner as an observer to third-party communication streams. This paper describes ongoing work that seeks to insert itself into that communication loop, fusing openly available data with requested content that is representative of what is sent to second parties. The mechanism for performing this is based on the sharing of falsified personal information through one-time online transactions that facilitate signup for newsletters, establish online accounts, or otherwise interact with resources on the Internet. The work has resulted in the real-time Use and Abuse of Personal Information OSINT collection engine that can ingest email, SMS text, and voicemail content at an enterprise scale. Foundations of this OSINT collection infrastructure are also laid to incorporate an artificial intelligence (AI)-driven interaction engine that shifts collection from a passive process to one that can effectively engage with different classes of content for improved real-world privacy experimentation and quantitative social science research.

Cyber influence defense: Applying the DISARM framework to a cognitive hacking case from the Romanian digital space

One of the main lessons learned in the context of Russia’s full-scale invasion of Ukraine starting in February 2022 is that for- eign information manipulation and interference (FIMI) operations are closely coupled with cyber threats. Regardless of whether cyberattacks are followed by an information manipulation compo- nent and vice versa, the merger of the two can be an early indica- tor of the potential for a conflict to escalate from the cyber area to the ground. Our article is premised on the idea that today’s highly technologised information ecosystem is a fertile ground for cyberattacks and information manipulation in the context of FIMI; more specifically, it enables cognitive hacking, meaning hacking the human mind and human cognition altogether through techno- logical disruption and cyber pressure. Starting from this premise, the aim of the article is to highlight the technological determi- nants of cognitive hacking and identify silent or emerging threats that bypass technological sensors and seek to disrupt and manip- ulate the information environment. The empirical part is based on observation as a descriptive method, which is used to analyse a case of cognitive hacking carried out via a YouTube malvertis- ing campaign targeting Romanian users. This case study is anal- ysed qualitatively by matching the DISinformation Analysis & Risk Management (DISARM) framework with evidence collected through Open-Source Intelligence (OSINT) tools, following an innovative analysis structured according to the purposes, actions, results and techniques (PART) model. The extensive analysis of the identified case shows that applying the DISARM framework to cyber-enabled operations can be useful for anticipating and responding to FIMI threats, even when such operations do not appear to have a spe- cific, immediately identifiable purpose.

Digitalisation of the Preliminary Investigation Phase, Fundamental and Human Rights and the Principle of Openness - Balancing Conflicting Interests in the Review of Large Data Sets

The EU courts have divided an investigation into two distinct stages with different aims: the preliminary investigation phase and the contradictory phase. This paper examines issues related to the digitalisation of the preliminary investigation phase, from screening and open source intelligence to data processing during unannounced inspections or dawn raids. The question is how rights of defence are secured without jeopardising an investigation where data sets have grown beyond anything previously known. Within the context of the principle of openness of government activities in Finland and Sweden, the author sets out to find how the main rule of openness is balanced with the objectives of the preliminary investigation phase. The article examines case-law and literature, complemented with public statements from competition authorities, to find that a fair balance between conflicting interests has been achieved thus far. Examples of open questions currently subject to debate do, nonetheless, range from using personal apps for detection to whether national identity as per Article 4(2) of the Treaty on European Union can tip the balance between confidentiality of correspondence and cartel enforcement categorically in favour of the former. The Court of Justice will likely have to address the issue of national identity in the ongoing Ronos case.

Grey literature in the intelligence domain: twilight or revival?

ABSTRACT Grey literature encompasses documentary material that is not commercially produced and includes a large and heterogeneous body of sources such as technical reports, working papers, business documents, and conference proceedings. Intelligence organisations worldwide have long incorporated grey literature in their collection and analysis activities as a cost-effective and indispensable source of valuable information that provides strategic insights and helps to form operational decisions. Despite advancements in electronic collection methods, the sourcing of grey literature in many instances still requires the physical presence of a collector, especially in countries and regions with low informational availability, limited digital infrastructure, and those ruled by authoritarian regimes. The classification of grey literature within other ‘INTs’ in general and open source intelligence (OSINT) in particular poses a challenge as it blurs the boundaries between human intelligence (HUMINT) and OSINT. Outsourcing OSINT collection, analysis, and dissemination to private vendors has been gaining speed and volume over the last decade. The article identifies three categories of private vendors active in collecting and analysing grey literature for the intelligence community while seeking to draw renewed attention to the importance of this source of information.

Strategic Review of the Effectiveness of Open-Source Intelligence at the National Maritime Information Center

This research provides a qualitative analysis of having Open-Source Intelligence (OSINT) at the National Maritime Information Center (NMIC). Indonesia's geographic condition demands strong marine defense and has enormous challenges in the maritime sector, so it is necessary to integrate maritime information data systems through the National Maritime Information Center. Technological advances provide opportunities and threats related to the potential to obtain information quickly. New technologies help localize, process, and disseminate large amounts of data and prepare it for analysis coming from Open-Source Intelligence. NMIC is required to be able to meet the needs of its users in the form of short and fast intelligence reports by utilizing information from OSINT. This is what underlies researchers to conduct research with problem formulations that have the effectiveness of OSINT in the NMIC. This research uses a qualitative method with an exploratory approach to find and analyze more in-depth things that are not yet known about OSINT. Productivity theory is used by researchers to discuss problem formulations assisted by NVIVO 12 tools in processing data and Soft System Methodology (SSM) to analyze data. The results showed that to use OSINT effectively, NMIC must have large data analysis capabilities using artificial intelligence assistance.

Improving quality of indicators of compromise using STIX graphs

Cybersecurity relies on Indicators of Compromise (IoCs) to detect and address threats. Although Threat Intelligence Platforms (TIPs) and Open Source Intelligence (OSINT) are common sources for gathering IoCs, their reliability varies. In our study, we enhance the management of IoCs and OSINT by introducing a novel method that reliably assesses IoC’s threat severity and confidence scores, focusing on Structured Threat Information eXpression (STIX) for threat associations. Our approach, implemented on OpenCTI, significantly enhances IoC value, as it aggregates threat intelligence from diverse sources utilizing a STIX graph-based approach, which is a unique feature among TIPs. Additionally, our method employs heuristic analysis to optimize IoC scoring. It takes into account factors such as relevance, completeness, timeliness, accuracy, and consistency while emphasizing the confidence of the source. Notably, the proposed method has enhanced the precision of the confidence score, achieving a 25.18% reduction in the average difference of confidence scores compared to the benchmarked platform. The Emotet and Medusa case studies underscore the importance of source credibility in confidence scores, emphasizing our TIP’s precision in cybersecurity threat assessment and defense enhancement.

Chaos in Order: Applying ML, NLP, and Chaos Theory in Open Source Intelligence for Counter-Terrorism

The present research aims to investigate whether Chaos Theory can be combined with Machine Learning and Natural Language Processing to apply these techniques to Open Source Intelligence (OSINT) analysis. Describing the role of OSINT in different domains and highlighting chaos as a valuable resource for information gathering, the study highlights that the substantial volume, swift velocity, and extensive variety of open-source data pose significant challenges. To address these challenges it is proposed to apply elements of Chaos Theory and advanced computational methods to open-source data. Key concepts from Chaos Theory that will be explored are the ‘Butterfly Effect’, and ‘Strange Attractors’, attempting to demonstrate that chaotic aspects of data can be exploited and transformed into dynamic and powerful sources of information. To support the above, the research includes a case study that exploits and analyses data from Reddit posts and concludes that recognizing and exploiting the dynamic interaction between order and chaos places Chaos Theory not only complementary but as a foundational stone of the overall OSINT toolkit, in the hands of intelligence analysts.

Using the case method for teaching information security students in open source intelligence

In today’s world, information is one of the most valuable resources, and the ability to process it becomes especially important in the process of training specialists of various profiles, including specialists in the field of information security. OSINT (Open Source Intelligence) is a technology for collecting and analyzing information about a person, organization, event or phenomenon from open sources. This powerful tool for the effective processing of open data is of particular interest for the formation of vocational competencies of students studying in the direction of training 10.03.01 “Information Security”.The authors of the article explore and test the principles of the case method as an effective method of teaching OSINT technologies. The case method, or the method of situational analysis, involves solving real-life problem situations and practical examples for deep understanding and mastering of the material. The article presents in detail specific cases on OSINT that can be used to teach students of “Information Security”. These tasks contribute to the development of critical thinking, the formation of analytical skills, and the ability to apply the acquired knowledge in practice.The article emphasizes the importance of feedback as a critical element of learning, which allows to customize and adapt the learning process according to students’ needs and level of mastery of OSINT technologies. In addition, the authors emphasize the importance of the multidisciplinary approach in OSINT training, including not only the study of technical aspects but also legal and ethical norms. The analysis of case method principles and case studies presented in the article will be useful for instructors developing OSINT training courses, as well as for specialists interested in modern methods and approaches to training in the field of information security.

Implications of Large Language Models for OSINT: Assessing the Impact on Information Acquisition and Analyst Expertise in Prompt Engineering

This paper explores the potential use of large language models (LLMs) in Open Source Intelligence (OSINT), with a focus on integrating information acquisition and the increasing importance of prompt engineering for analysts. The research includes a comprehensive literature review, which highlights the widespread use of AI in OSINT and the related challenges, such as data validity and ethical concerns. The study emphasizes the significance of prompt engineering as a crucial skill that demands a profound comprehension of LLMs to generate validated intelligence. A model of the OSINT lifecycle that incorporates LLMs is proposed. The paper further discusses updated training in critical thinking, search techniques, and prompt engineering for intelligence professionals. The findings indicate a noteworthy shift in OSINT procedures, highlighting the importance of continuous research and education to fully utilize AI in intelligence gathering.

Breaking the trend: Anomaly detection models for early warning of socio-political unrest

This paper presents an innovative Early Warning System for predicting conflicts and unrest based on Anomaly Detection, identifying sudden and unexpected changes in behavioral patterns that may indicate the potential for these events to occur. This approach draws inspiration from various fields – including industry, such as manufacturing, physics and networking – but its application in the domain of diplomacy is entirely new. The system, tested on three case studies, showcase its ability to enhance open-source intelligence technique in the diplomatic arena. The study provides a fresh perspective on predictive analytics and focuses on examining outbreaks.

A systematic review on research utilising artificial intelligence for open source intelligence (OSINT) applications

This paper presents a systematic review to identify research combining artificial intelligence (AI) algorithms with Open source intelligence (OSINT) applications and practices. Currently, there is a lack of compilation of these approaches in the research domain and similar systematic reviews do not include research that post dates the year 2019. This systematic review attempts to fill this gap by identifying recent research. The review used the preferred reporting items for systematic reviews and meta-analyses and identified 163 research articles focusing on OSINT applications leveraging AI algorithms. This systematic review outlines several research questions concerning meta-analysis of the included research and seeks to identify research limitations and future directions in this area. The review identifies that research gaps exist in the following areas: Incorporation of pre-existing OSINT tools with AI, the creation of AI-based OSINT models that apply to penetration testing, underutilisation of alternate data sources and the incorporation of dissemination functionality. The review additionally identifies future research directions in AI-based OSINT research in the following areas: Multi-lingual support, incorporation of additional data sources, improved model robustness against data poisoning, integration with live applications, real-world use, the addition of alert generation for dissemination purposes and incorporation of algorithms for use in planning.

Securing the Web

In an era characterized by the ubiquity of the internet, the proliferation of online services, and the increasing frequency of cyber threats, the detection of look-alike domains has become a critical component of cybersecurity. The current paper presents an approach for the detection of look-alike domains, leveraging the power of open-source intelligence (OSINT) tools. It included gathering and analyzing a wide range of publicly available data sources, including permutations, WHOIS records, IP information, website content, Geo IP, similarity percentage, name server, and mail server records, and building a comprehensive profile of domains under investigation. Through the application of online search engines, patterns and features that distinguish legitimate domains from their deceptive counterparts were established. The analysis demonstrated that OSINT tools provided significant information about the sample domains and successfully detected 1598 registered look-alike domains among 10 sample domains using dnstwist, while OpenSquat identified 103 squatting domains, 960 active phishing websites, and 53 domains with suspicious certificates across five sample websites. The research contributes to the enhancement of cybersecurity practices by providing a cost-effective and scalable solution for identifying look-alike domains, which can serve as precursors to various online threats, including phishing attacks, malware distribution, and fraud.