Cyber Threat Intelligence Research Articles

Encrypted video traffic clustering demystified

Cyber threat intelligence officers and forensics investigators often require the behavioural profiling of groups based on their online video viewing activity. It has been demonstrated that encrypted video traffic can be classified under the assumption of using a known subset of video titles based on temporal video viewing trends of particular groups. Nonetheless, composing such a subset is extremely challenging in real situations. Therefore, this work exhibits a novel profiling scheme for encrypted video traffic with no a priori assumption of a known subset of titles. It introduces a seminal synergy of Natural Language Processing (NLP) and Deep Encoder-based feature embedding algorithms with refined clustering schemes from off-the-shelf solutions, in order to group viewing profiles with unknown video streams. This study is the first to highlight the most computationally effective, accurate combinations of feature embedding and clustering using real datasets, thereby, paving the way to future forensics tools for automated behavioural profiling of malicious actors.

A Comparative Analysis of Cyber-Threat Intelligence Sources, Formats and Languages

The sharing of cyber-threat intelligence is an essential part of multi-layered tools used to protect systems and organisations from various threats. Structured standards, such as STIX, TAXII and CybOX, were introduced to provide a common means of sharing cyber-threat intelligence and have been subsequently much-heralded as the de facto industry standards. In this paper, we investigate the landscape of the available formats and languages, along with the publicly available sources of threat feeds, how these are implemented and their suitability for providing rich cyber-threat intelligence. We also analyse at a sample of cyber-threat intelligence feeds, the type of data they provide and the issues found in aggregating and sharing the data. Moreover, the type of data supported by various formats and languages is correlated with the data needs for several use cases related to typical security operations. The main conclusions drawn by our analysis suggest that many of the standards have a poor level of adoption and implementation, with providers opting for custom or traditional simple formats.

TIMiner: Automatically extracting and analyzing categorized cyber threat intelligence from social data

Security organizations increasingly rely on Cyber Threat Intelligence (CTI) sharing to enhance resilience against cyber threats. However, its effectiveness remains dubious due to two major limitations: first, the existing approaches fail to identify the unseen types of Indicator of compromise (IOC); second, they are incapable of automatically generating categorized CTIs with domain tags (e.g., finance, government), which makes CTI sharing ineffective. To combat the challenges, this paper proposes TIMiner, a novel automated framework for CTI extraction and sharing based on social media data. Particularly, an efficient domain recognizer based on convolutional neural network is first implemented to identify CTIs’ targeted domain. Then, an indicator of compromise (IOC) extraction approach based on word embedding and syntactic dependence is proposed, which provides the ability to identify unseen types of IOCs. Finally, the extracted IOC and its domain tag are integrated to generate a categorized CTI with specific-domain. TIMiner is capable of generating CTIs with domain tags automatically. With the categorized CTIs, Threat-Index is presented to quantify the severity of the threats toward different domains. Experimental results confirm that the proposed CTI domain recognizer and IOC extraction achieve superior performance with the accuracy exceeding 84% and 94%, respectively. Moreover, TIMiner stimulates new insights on the evolution of cyber attacks across multiple domains.

Automatic extraction of named entities of cyber threats using a deep Bi-LSTM-CRF network

Countless cyber threat intelligence (CTI) reports are used by companies around the world on a daily basis for security reasons. To secure critical cybersecurity information, analysts and individuals should accordingly analyze information on threats and vulnerabilities. However, analyzing such overwhelming volumes of reports requires considerable time and effort. In this study, we propose a novel approach that automatically extracts core information from CTI reports using a named entity recognition (NER) system. During the process of constructing our proposed NER system, we defined meaningful keywords in the security domain as entities, including malware, domain/URL, IP address, Hash, and Common Vulnerabilities and Exposures. Furthermore, we linked these keywords with the words extracted from the text data of the report. To achieve a higher performance, we utilized the character-level feature vector as an input to bidirectional long-short-term memory using a conditional random field network. We finally achieved an average F1-score of 75.05%. We release 498,000 tag datasets created during our research.

Computer Forensic Methodology and Tools

The Computer Forensic has long played a vital role across the domains - Cyber Crimes Investigations, Incident Response, Cyber Threat Intelligence and Corporate Cybersecurity. In the past Digital Evidence lost their validity in the Courtrooms. Prior, similar to some other Physical Evidence, courts used to consider evidence procured from computer systems as genuine. In any case, as time passed, we figured out the fact that it is so natural to corrupt, destroy, or alter computer data, all three stages - storage, network and computation. In the least complex model, if an individual basically opens a computer file, it's absolutely impossible to demonstrate the last date that the document was refreshed aka updated. Computers are intended to record the time and date of an accessed file on its own by default features of an operating system. Examiners later understood that they have to build up the essential devices and procedures to separate the necessary data from computers without influencing the data all the while. With time, specialists concocted systems to securely recover data, preserve and analyze to derive the insights useful in Computer related Crimes. This part of forensic science is currently well known as Computer Forensics.

Customer-oriented ranking of cyber threat intelligence service providers

• A novel framework for Cyber Threat Intelligence (CTI) service provider selection. • Ranks CTI service provider according to the security requirements of consumer. • A first of its kind that identifies a comprehensive selection criteria, i.e., Key Performance Indicators for ranking CTI service providers. • Employs multi-criteria decision making technique for ranking CTI service provider. Cyber Threat Intelligence (CTI) is revolutionizing the cybersecurity ecosystem by enabling organizations to share threat data and allow for proactive defense against sophisticated intrusion attempts. Several cybersecurity vendors are offering a variety of CTI services and capabilities to their customers. The wide variety in the CTI service offerings by different providers makes it difficult for the security experts to decide which service provider is most suitable according to their security program requirements. Current approaches for ranking CTI service providers are based on a limited set of Key Performance Indicators (KPI) and do not consider the customer’s security program relevance and requirements. In this work, an extensive set of KPI to rank CTI service providers is defined. This paper proposes a framework, which is a first of its kind that assigns weights to KPI based on the customer’s requirements and ranks CTI service providers. This framework significantly helps the security program leaders in decision making for CTI service provider selection. It is designed to improve the quality of service offerings by promoting healthy competition among CTI service providers.

BLOCIS: Blockchain-Based Cyber Threat Intelligence Sharing Framework for Sybil-Resistance

The convergence of fifth-generation (5G) communication and the Internet-of-Things (IoT) has dramatically increased the diversity and complexity of the network. This change diversifies the attacker’s attack vectors, increasing the impact and damage of cyber threats. Cyber threat intelligence (CTI) technology is a proof-based security system which responds to these advanced cyber threats proactively by analyzing and sharing security-related data. However, the performance of CTI systems can be significantly compromised by creating and disseminating improper security policies if an attacker intentionally injects malicious data into the system. In this paper, we propose a blockchain-based CTI framework that improves confidence in the source and content of the data and can quickly detect and eliminate inaccurate data for resistance to a Sybil attack. The proposed framework collects CTI by a procedure validated through smart contracts and stores information about the metainformation of data in a blockchain network. The proposed system ensures the validity and reliability of CTI data by ensuring traceability to the data source and proposes a system model that can efficiently operate and manage CTI data in compliance with the de facto standard. We present the simulation results to prove the effectiveness and Sybil-resistance of the proposed framework in terms of reliability and cost to attackers.

SCERM—A novel framework for automated management of cyber threat response activities

Cyber Threat Management (CTM) involves prevention, detection, and response to cyber-attacks by identifying and understanding threats, and applying appropriate actions. This is not practical for an organization to perform these activities within the time-frame of an impending attack. Organizations should swiftly accumulate and share Cyber Threat Intelligence (CTI) with peers to make effective use of shared threat information. Efforts are underway for standardizing the expression of threats into a machine-understandable format. Structured Threat Information eXpression (STIX) is a comprehensive effort that structures CTI, enables its sharing, visualization, and analysis. Although a large volume of STIX reports is available publicly, their state remains poor. Reports are not appropriately formatted, use incorrect vocabulary, and mislabel or omit key components, which curtail their usefulness for effective cyber threat management. For a meaningful analysis, an analyst needs a curated document list categorized according to cyber threat management phases for the under-investigation threat. We believe that methods for valuation of structured threat documents based on cyber threat management phases are limited or non-existent. We present a novel framework named SCERM—Structured threat data Cleansing, Evaluation, and Refinement. SCERM formally models the STIX architecture and valuates reports on the basis of the use case “managing cyber threat response activities”. It uplifts CTI by remapping wrongly placed contents to the STIX data model. SCERM refines incomplete or missing components through a pre-prepared dataset of curated blog reports. This process is repeated until the reports improve to a threshold suitable for cyber threat management. A case study is presented to demonstrate the working of SCERM. The evaluation valuates publicly available STIXs for cyber threat management. It is observed that current STIX reports have limited information on prevention and almost none for the response phase of cyber threat management. The results demonstrate that SCERM significantly enriches STIX reports. The improvement in prevention is 73% and in the response is a 100%.

Measuring and visualizing cyber threat intelligence quality

The very raison d’être of cyber threat intelligence (CTI) is to provide meaningful knowledge about cyber security threats. The exchange and collaborative generation of CTI by the means of sharing platforms has proven to be an important aspect of practical application. It is evident to infer that inaccurate, incomplete, or outdated threat intelligence is a major problem as only high-quality CTI can be helpful to detect and defend against cyber attacks. Additionally, while the amount of available CTI is increasing it is not warranted that quality remains unaffected. In conjunction with the increasing number of available CTI, it is thus in the best interest of every stakeholder to be aware of the quality of a CTI artifact. This allows for informed decisions and permits detailed analyses. Our work makes a twofold contribution to the challenge of assessing threat intelligence quality. We first propose a series of relevant quality dimensions and configure metrics to assess the respective dimensions in the context of CTI. In a second step, we showcase the extension of an existing CTI analysis tool to make the quality assessment transparent to security analysts. Furthermore, analysts’ subjective perceptions are, where necessary, included in the quality assessment concept.

AI4SAFE-IoT: an AI-powered secure architecture for edge layer of Internet of things

With the increasing use of the Internet of things (IoT) in diverse domains, security concerns and IoT threats are constantly rising. The computational and memory limitations of IoT devices have resulted in emerging vulnerabilities in most IoT-run environments. Due to the low processing ability, IoT devices are often not capable of running complex defensive mechanisms. Lack of an architecture for a safer IoT environment is referred to as the most important barrier in developing a secure IoT system. In this paper, we propose a secure architecture for IoT edge layer infrastructure, called AI4SAFE-IoT. This architecture is built upon AI-powered security modules at the edge layer for protecting IoT infrastructure. Cyber threat attribution, intelligent web application firewall, cyber threat hunting, and cyber threat intelligence are the main modules proposed in our architecture. The proposed modules detect, attribute, and further identify the stage of an attack life cycle based on the Cyber Kill Chain model. In the proposed architecture, we define each security module and show its functionality against different threats in real-world applications. Moreover, due to the integration of AI security modules in a different layer of AI4SAFE-IoT, each threat in the edge layer will be handled by its corresponding security module delivered by a service. We compared the proposed architecture with the existing models and discussed our architecture independence of the underlying IoT layer and its comparatively low overhead according to delivering security as service for the edge layer of IoT architecture instead of embed implementation. Overall, we evaluated our proposed architecture based on the IoT service management score. The proposed architecture obtained 84.7 out of 100 which is the highest score among peer IoT edge layer security architectures.

Analysis of adversary activities using cloud-based web services to enhance cyber threat intelligence

The understanding of cyber threats to a network is challenging yet rewarding as it allows an organisation to prevent a potential attack. Numerous efforts have been made to predict cyber threat before they occur. To build a threat intelligence framework, an organisation must understand attack data collected from the network events and analyse them to identify the cyber attack artefacts such as IP address, domain name, tools and techniques, username and password, and geographic location of the attacker, which could be used to understand the nature of attack to a system or network. However, it is very difficult or dangerous to collect and analyse live data from a production system. Honeypot technology is well known for mimicking the real system while collecting actual data that can be in near real time in order to monitor the activities on the network. This paper proposes a threat intelligence approach analysing attack data collected using cloud-based web service in order to support the active threat intelligence.

An Attribution of Cyberattack using Association Rule Mining (ARM)

With the rapid development of computer networks and information technology, an attacker has taken advantage to manipulate the situation to launch a complicated cyberattack. This complicated cyberattack causes a lot of problems among the organization because it requires an effective cyberattack attribution to mitigate and reduce the infection rate. Cyber Threat Intelligence (CTI) has gain wide coverage from the media due to its capability to provide CTI feeds from various data sources that can be used for cyberattack attribution. In this paper, we study the relationship of basic Indicator of Compromise (IOC) based on a network traffic dataset from a data mining approach. This dataset is obtained using a crawler that is deployed to pull security feed from Shadowserver. Then an association analysis method using Apriori Algorithm is implemented to extract rules that can discover interesting relationship between large sets of data items. Finally, the extracted rules are evaluated over the factor of interestingness measure of support, confidence and lift to quantify the value of association rules generated with Apriori Algorithm. By implementing the Apriori Algorithm in Shadowserver dataset, we discover some association rules among several IOC which can help attribute the cyberattack.

Adversarial Active Learning for Named Entity Recognition in Cybersecurity

Owing to the continuous barrage of cyber threats, there is a massive amount of cyber threat intelligence. However, a great deal of cyber threat intelligence come from textual sources. For analysis of cyber threat intelligence, many security analysts rely on cumbersome and time-consuming manual efforts. Cybersecurity knowledge graph plays a significant role in automatics analysis of cyber threat intelligence. As the foundation for constructing cybersecurity knowledge graph, named entity recognition (NER) is required for identifying critical threat-related elements from textual cyber threat intelligence. Recently, deep neural network-based models have attained very good results in NER. However, the performance of these models relies heavily on the amount of labeled data. Since labeled data in cybersecurity is scarce, in this paper, we propose an adversarial active learning framework to effectively select the informative samples for further annotation. In addition, leveraging the long short-term memory (LSTM) network and the bidirectional LSTM (BiLSTM) network, we propose a novel NER model by introducing a dynamic attention mechanism into the BiLSTM-LSTM encoderdecoder. With the selected informative samples annotated, the proposed NER model is retrained. As a result, the performance of the NER model is incrementally enhanced with low labeling cost. Experimental results show the effectiveness of the proposed method.

Guest Editorial Special Issue on Big Data Applications and Techniques in Cyber Threat Intelligence

Guest Editorial Special Issue on Big Data Applications and Techniques in Cyber Threat Intelligence

From logs to Stories: Human-Centred Data Mining for Cyber Threat Intelligence

An average medium-sized organisation logs approx. 10 to 500 mln events per day on the system. Only less than 5% of threat alerts are being investigated by the specialised staff, leaving the security hole open for potential attacks. Insufficient information in alert message produced in machine-friendly rather than human-friendly format causes cognitive overload on currently limited cybersecurity resources. In this paper, the model that generates the report in natural language by means of applying novel storytelling techniques from security logs is proposed. The solution caters for different levels of reader expertise and preference by providing adjustable templates, filled from both local and global knowledge base. The validation is performed on case study from Security Operations Centre (SOC) at educational institution. The report generated proves superior to existing approach in terms of comprehension (increased cognition) and completeness (enriched context). The evaluation demonstrates power of storytelling in potential threats interpretation in cybersecurity context.

Sharing of Cyber Threat Intelligence between States

Novel environmental invasive biotechnologies, such as gene drives and Horizontal Environmental Genetic Alteration Agents exceed the classical applications of genetically modified organisms. The reason for this is that they are designed to transform wild organisms into genetically modified organisms which express desired traits. Instead of in a laboratory, this transformation takes place in the environment. The far-ranging effects that may be triggered by gene drive and Horizontal Environmental Genetic Alteration Agents require an extension of risk assessment to include socio-political consequences. The present article offers a first brief examination whether regulation is prepared for possible conflicts caused by benevolent or by hostile use of these novel technologies.

Basic model of information processes and behavior of a cyber defense system

Increasing the complexity and intensity of cyberattacks makes the actual task of implementing a proactive strategy for protecting technology systems (ITS). Such a strategy is associated with tight time limits on making decisions on assessing the current situation and making appropriate decisions even before the interruption of a cyber attack. These limitations significantly narrow the scope of the use of expert assessment methods. There is a growing need for the widespread use of automation tools that will significantly reduce the time for determining security events, and for formulating and implementing a decision on countering the invasion of ITS cyberspace. A feature of such tools is a significant level of intellectualization, which is adequate to the level of competence of cybersecurity operators. The basic component of the procedure for developing automation tools is the formation of a process model of the object of research. Based on this modem, using the well-known technological platforms, the corresponding software code is generated. Known means of a formalized description of business processes make it possible to form models of processes. The term “information” is widely used, but its essence is not disclosed. This, in turn, leads to a number of regarding the properties of these processes (composition, structure, functions, organization, etc.). The lack of a constructive definition of the term “information” greatly complicates the analysis of cyber defense processes. In cyberspace, security events and security are procedures for processing, storing and transmitting data (bit sets). Under these conditions, it is difficult to unambiguously express the semantic connection between the dangerous event in ITS, the objects reflecting this event, and cyber defense processes using existing modeling tools. The problem of information uncertainties in modeling a cyber defense system significantly narrows the range of processes that can be algorithmized with the aim of creating means for their automation. The solution to this problem is relevant for the sphere of automation of proactive cyber defense system processes. In order to increase the efficiency of the process of developing automation tools for modern cyber defense systems, a new model of such a system has been developed based on an attribute-transfer approach to the essence of and a basic model of processes for managing a cyber system. Within the framework of this model, the IT system is a management object, the security of which is controlled by a set of coordinated cyber protection processes. This set of processes is connected by a control cycle. The sequence of cycles is the trajectory of the cyber defense system. Corresponding graphic and mathematical models of behavior are developed. Using them, a decomposition of one of the cyber threat intelligence processes was carried out and an appropriate automation tool was developed.

Special Issue on Big Data Applications in Cyber Security and Threat Intelligence – Part 2

The papers in this special section focus on Big Data applications in cybersecurity and threat intelligence. The last decade has witnessed a tremendous rapid increase in volume, veracity, velocity and variety of data (also commonly referred to as the four V’s of big data in the literature1) generated by different cyber security solutions and as part of cyber investigation cases. When a significant amount of data is collected from or generated by different devices and sources, intelligent big-data analytical techniques are necessary to mine, interpret and visualize such data. To mitigate existing cyber security threats, it is important for big-data analytical techniques to keep pace. Therefore, in special issue we focus on cutting-edge from both academia and industry, with a particular emphasis on novel techniques to mine, interpret and visualize big-data from a wide range of sources and can be applied in cyber security, cyber forensics and threat intelligence context.

Big Data Sanitization and Cyber Situational Awareness: A Network Telescope Perspective

This paper addresses the problems of data sanitization and cyber situational awareness by analyzing 910 GB of real Internet-scale traffic, which has been passively collected by monitoring close to 16.5 million darknet IP addresses from a /8 and a /13 network telescopes. First, the paper offers a novel probabilistic darknet preprocessing model, which aims at sanitizing darknet data to prepare it for effective use in the task of cyber threat intelligence generation. Such model has been engineered using a distributed multithreaded approach, rendering it operational and highly effective on darknet big data. Second, the paper further contributes by presenting an innovative approach to infer large-scale orchestrated probing campaigns by leveraging darknet data, for Internet cyber situational awareness. The approach uniquely reduces the dimensionality of such big data by utilizing its artifacts, instead of processing the actual raw data. This is accomplished by extracting and analyzing probing time series using formal methods rooted in Fourier transform and Kalman filtering. Thorough empirical evaluations indeed validate the accuracy and the performance of the proposed methods and techniques. We assert that the darknet sanitization model and the probing orchestration inference approach are of significant value, given their postulated highly applicable nature to the field of Internet measurements for cyber security in the era of big data.

Cyber Threat Intelligence for Improving Cybersecurity and Risk Management in Critical Infrastructure

Cyber-attack is one of the significant threats affecting to any organisation specifically to the Critical Infrastructure (CI) organisation. These attacks are nowadays more sophisticated, multi-vectored and less predictable, which make the Cyber Security Risk Management (CSRM) task more challenging. Critical Infrastructure needs a new line of security defence to control these threats and minimise risks. Cyber Threat Intelligence (CTI) provides evidence-based information about the threats aiming to prevent threats. There are existing works and industry practice that emphasise the necessity of CTI and provides methods for threat intelligence and sharing. However, despite these significant efforts, there is a lack of focus on how CTI information can support the CSRM activities so that the organisation can undertake appropriate controls to mitigate the risk proactively. This paper aims to fill this gap by integrating CTI for improving cybersecurity risks management practice specifically focusing on the critical infrastructure. In particular, the proposed approach contributes beyond state of the art practice by incorporating CTI information for the risk management activities. This helps the organisation to provide adequate and appropriate controls from strategic, tactical and operational perspectives. We have integrated concepts relating to CTI and CSRM so that threat actor's profile, attack detailed can support calculating the risk. We consider smart grid system as a Critical Infrastructure to demonstrate the applicability of the work. The result shows that cyber risks in critical infrastructures can be minimised if CTI information is gathered and used as part of CSRM activities. CTI not only supports understanding of threat for accurate risk estimation but also evaluates the effectiveness of existing controls and recommend necessity controls to improve overall cybersecurity. Also, the result shows that our approach provides early warning about issues that need immediate attention.