Alert Correlation Research Articles

Semantic aware attack scenarios reconstruction

Intrusion analysis is a resource intensive, complex and expensive process for any organization. The reconstruction of the attack scenario is an important aspect of such endeavor. We tackle in this paper several challenges overlooked by existing attack scenarios reconstruction techniques that undermine their performances. These include the ability to identify and extract novel attack patterns and the correlation of heterogeneous multisensor alerts. We propose a novel attack scenario reconstruction approach that analyzes both implicit and explicit relationships between intrusion alerts using semantic analysis and a new intrusion ontology. The proposed approach can reconstruct known and unknown attack scenarios and correlate alerts generated in multi-sensor IDS environment. Moreover, our approach can handle for the first time both novel attacks and false negative alerts generated by Intrusion Detection Systems (IDSs). Our experimental results show the potential of our approach and its advantages over previous approaches.

Alert Correlation through a Multi Components Architecture

Alert correlation is a process that analyzes the raw alerts produced by one or more intrusion detection systems, reduces nonrelevant ones, groups together alerts based on similarity and causality relationships between them and finally makes aconcise and meaningful view of occurring or attempted intrusions. Unfortunately, most correlation approaches use just a few components that aim only specific correlation issues and so cause reduction in correlation rate. This paper uses a general correlation model that has already been presented in [9] and is consisted of a comprehensive set of components. Then some changes are applied in the component that is related to multi-step attack scenario to detect them better and so to improve semantic level of alerts. The results of experiments with DARPA 2000 data set obviously show the effectiveness of the proposed approach. DOI: http://dx.doi.org/10.11591/ijece.v3i4.2771

Using Artificial Immune System and Fuzzy Logic for Alert Correlation.

One of the most important challenges facing the intrusion detection systems (IDSs) is the huge number of generated alerts. A system administrator will be overwhelmed by these alerts in such a way that she/he cannot manage and use the alerts. The best-known solution is to correlate low-level alerts into a higher level attack and then produce a high-level alert for them. In this paper a new automated alert correlation approach is presented. It employs Fuzzy Logic and Artificial Immune System (AIS) to discover and learn the degree of correlation between two alerts and uses this knowledge to extract the attack scenarios. The proposed system doesn't need vast domain knowledge or rule definition efforts. To correlate each new alert with previous alerts, the system first tries to find the correlation probability based on its fuzzy rules. Then, if there is no matching rule with the required matching threshold, it uses the AIRS algorithm. The system is evaluated using DARPA 2000 dataset and a netForensics honeynet data. The completeness, soundness and false alert rate are calculated. The average completeness for LL-DoS1.0 and LLDoS2.0, are 0.957 and 0.745 respectively. The system generates the attack graphs with an acceptable accuracy and, the computational complexity of the probability assignment algorithm is linear.

High-quality attack graph-based IDS correlation

Intrusion Detection Systems are widely deployed in computer networks. As modern attacks are getting more sophisticated and the number of sensors and network nodes grow, the problem of false positives and alert analysis becomes more difficult to solve. Alert correlation was proposed to analyse alerts and to decrease false positives. Knowledge about the target system or environment is usually necessary for efficient alert correlation. For representing the environment information as well as potential exploits, the existing vulnerabilities and their Attack Graph (AG) is used. It is useful for networks to generate an AG and to organize certain vulnerabilities in a reasonable way. In this article, a correlation algorithm based on AGs is designed that is capable of detecting multiple attack scenarios for forensic analysis. It can be parameterized to adjust the robustness and accuracy. A formal model of the algorithm is presented and an implementation is tested to analyse the different parameters on a real set of alerts from a local network. To improve the speed of the algorithm, a multi-core version is proposed and a HMM-supported version can be used to further improve the quality. The parallel implementation is tested on a multi-core correlation platform, using CPUs and GPUs.

Fast attack detection using correlation and summarizing of security alerts in grid computing networks

Due to the extensive growth of grid computing networks, security is becoming a challenge. Usual solutions are not enough to prevent sophisticated attacks fabricated by multiple users especially when the number of nodes connected to the network is changing over the time. Attackers can use multiple nodes to launch DDoS attacks which generate a large amount of security alerts. On the one hand, this large number of security alerts degrades the overall performance of the network and creates instability in the operation of the security management solutions. On the other hand, they can help in camouflaging other real attacks. To address these issues, a correlation mechanism is proposed which reduces the security alerts and continue detecting attacks in grid computing networks. To obtain the more accurate results, a major portion of the experiments are performed by launching DDoS and Brute Force (BF) attacks in real grid environment, i.e., the Grid'5000 (G5K) network.

Alert correlation using artificial immune recognition system

High volumes of low-level alerts that are generated by intrusion detection systems (IDSs) are serious obstacle for using them effectively. These high volumes of alerts overwhelm system administrators in such a way that they cannot manage and interpret them. Alert correlation is used to reduce the number of alerts and increase their level of abstraction. It selects a group of low-level alerts and converts them into a higher level attack and then produces a high-level alert for them. In this paper, a new artificial immune system-based alert correlation system is presented, named AISAC. It learns the correlation probability between each pair of alert types and uses this knowledge to extract the attack scenarios. AISAC does not need intensive domain knowledge and rule definition efforts. It also does not need to manually update the extracted knowledge. The computational cost of learning algorithm is linear, and the initial learning is done by a very limited general data in offline mode. AISAC is evaluated by DARPA 2000 and netForensics Honeynet data. Results show that although it uses a relatively simple algorithm, it generates the attack graphs with acceptable accuracy.

Alert correlation: Severe attack prediction and controlling false alarm rate tradeoffs

Alert correlation plays an increasingly crucial role in nowadays computer security infrastructures. It is particularly needed for coping with the huge amounts of alerts which are daily triggered by intrusion detection systems IDSs, fire-walls, etc. While the use of multiple IDSs, security tools and complementary approaches is fundamental and highly recommended in order to improve the overall detection rates, this however inevitably causes huge amounts of alerts most of which are redundant and false alarms making the manual analysis of these triggered alerts time-consuming and inefficient. This paper addresses three important issues related to predicting severe attacks attacks with high dangerousness levels by analyzing inoffensive and preparatory attacks. i Firstly, we address the issue of preprocessing alerts reported by the multiple detection tools in order to eliminate the redundant and irrelevant alerts and format them so that they can be analyzed by a severe attack prediction model. ii Then, we propose a novel prediction model based on a Bayesian network multi-net allowing on one hand to better model the severe attacks and on the other hand handle the reliability of IDSs when predicting severe attacks. iii Finally, we provide a flexible and efficient approach especially designed to limit the false alarm rates by controlling the confidence of the prediction model. The main benefits of our approach is an integrated model guaranteeing very promising prediction/false alarm rate tradeoffs with minimum expert intervention. Our experimental studies are carried out on a real and representative alert corpus generated by the de facto network-based IDS Snort, and show very interesting performances regarding the tradeoffs between the prediction rates and the corresponding false alarm ones.

An alert correlation platform for memory‐supported techniques

SUMMARYIntrusion Detection Systems (IDS) have been widely deployed in practice for detecting malicious behavior on network communication and hosts. False‐positive alerts are a popular problem for most IDS approaches. The solution to address this problem is to enhance the detection process by correlation and clustering of alerts. To meet the practical requirements, this process needs to be finished fast, which is a challenging task as the amount of alerts in large‐scale IDS deployments is significantly high. We identifytextitdata storage and processing algorithms to be the most important factors influencing the performance of clustering and correlation. We propose and implement a highly efficient alert correlation platform. For storage, a column‐based database, an In‐Memory alert storage, and memory‐based index tables lead to significant improvements of the performance. For processing, algorithms are designed and implemented which are optimized for In‐Memory databases, e.g. an attack graph‐based correlation algorithm. The platform can be distributed over multiple processing units to share memory and processing power. A standardized interface is designed to provide a unified view of result reports for end users. The efficiency of the platform is tested by practical experiments with several alert storage approaches, multiple algorithms, as well as a local and a distributed deployment. Copyright © 2011 John Wiley & Sons, Ltd.

A hybrid model for correlating alerts of known and unknown attack scenarios and updating attack graphs

Managing and analyzing a huge number of low-level alerts is very difficult and exhausting for network administrators. Alert correlation methods have been proposed to decrease the number of alerts and make them more intelligible. Proposed methods for alert correlation are different in terms of their performance, accuracy and adaptivity. We present a new hybrid model not only to correlate alerts as accurately and efficiently as possible but also to be able to boost the model in the course of time. The model presented in this paper consists of two parts: (1) an attack graph-based method to correlate alerts raised for known attacks and hypothesize missed alerts and (2) a similarity-based method to correlate alerts raised for unknown attacks which can not be correlated using the first part and also to update the attack graph. These two parts cooperate with each other such that if the first part could not correlate a new alert, the second part is applied. We propose two different methods for these two parts. In order to update the attack graph, we present a technique (using the similarity-based method in the second part of the model) which is actually the most salient feature of our model: capability of hypothesizing missed exploits and discovering defects in pre and post conditions of known exploits in attack graphs. We also propose an additional method named alerts bisimulation for compressing graphs of correlated alerts. The results of experiments on DARPA2000 clearly show the model can accurately correlate alerts. Also the ability of updating attack graphs is illustrated using an experiment.

Alert correlation in collaborative intelligent intrusion detection systems—A survey

As complete prevention of computer attacks is not possible, intrusion detection systems (IDSs) play a very important role in minimizing the damage caused by different computer attacks. There are two intrusion detection methods: namely misuse- and anomaly-based. A collaborative, intelligent intrusion detection system (CIIDS) is proposed to include both methods, since it is concluded from recent research that the performance of an individual detection engine is rarely satisfactory. In particular, two main challenges in current collaborative intrusion detection systems (CIDSs) research are highlighted and reviewed: CIDSs system architectures and alert correlation algorithms. Different CIDSs system, architectures are explained and compared. The use of CIDSs together with other multiple security systems raise certain issues and challenges in, alert correlation. Several different techniques for alert correlation are discussed. The focus will be on correlation of CIIDS alerts. Computational, Intelligence approaches, together with their applications on IDSs, are reviewed. Methods in soft computing collectively provide understandable, and autonomous solutions to IDS problems. At the end of the review, the paper suggests fuzzy logic, soft computing and other AI techniques, to be exploited to reduce the rate of false alarms while keeping the detection rate high. In conclusion, the paper highlights opportunities for an integrated solution to large-scale CIIDS.

An ontology-based intrusion alerts correlation system

Alert correlation techniques effectively improve the quality of alerts reported by intrusion detection systems, and are sufficient to support rapid identification of ongoing attacks or predict an intruder’s next likely goal. In our previous work, an alert correlation approach based on our XSWRL ontology has been proposed. This paper focuses on how to develop the intrusion alerts correlation system according to our alert correlation approach. At first, the multi-agent system architecture consisting of agents and sensors is shown. The sensors collect security relevant information, and the agents process the information. Then we present each modules of the system in detail. The State Sensor collects information about security state and the Local State Agent and Center State Agent preprocess the security state information and convert it to ontology. The Attack Sensor collects information about attack and the Local Alert Agent and Center Alert Agent preprocess the alert information and convert it to ontology. The Attack Correlator correlates the attacks and outputs the attack sessions.

An incremental frequent structure mining framework for real-time alert correlation

With the large volume of alerts produced by low-level detectors, management of intrusion alerts is becoming more challenging. Manual analysis of a large number of raw alerts is both time consuming and labor intensive. Alert Correlation addresses this issue by finding similarity and causality relationships between raw alerts to provide a condensed, yet more meaningful view of the network from the intrusion standpoint. While some efforts have been made in the literature by researchers to find the relationships between alerts automatically, not much attention has been given to the issue of real-time correlation of alerts. Previous learning-based approaches either fail to cope with a large number of generated alerts in a large-scale network or do not address the problem of concept drift directly. In this paper, we propose a framework for real-time alert correlation which incorporates novel techniques for aggregating alerts into structured patterns and incremental mining of frequent structured patterns. Our approach to aggregation provides a reduced view of developed patterns of alerts. At the core of the proposed framework is a new algorithm (FSP_Growth) for mining frequent patterns of alerts considering their structures. In the proposed framework, time-sensitive statistical relationships between alerts are maintained in an efficient data structure and are updated incrementally to reflect the latest trends of patterns. The results of experiments conducted with the DARPA 2000 dataset as well as artificial data clearly demonstrate the efficiency of proposed techniques. A promising reduction ratio of 96% is achieved on the DARPA 2000 dataset. The running time of the FSP_Growth algorithm scales linearly with the size of artificial datasets. Moreover, testing the proposed framework with alert logs of a real-world network shows its ability to extract interesting patterns among the alerts. The ability to answer useful time-sensitive queries regarding pattern co-occurrences is another advantage of the proposed method compared to other approaches.

Real-Time Alert Correlation with Type Graphs

The premise of automated alert correlation is to accept that false alerts from a low level intrusion detection system are inevitable and use attack models to explain the output in an understandable way. Several algorithms exist for this purpose which use attack graphs to model the ways in which attacks can be combined. These algorithms can be classified in to two broad categories namely scenario-graph approaches, which create an attack model starting from a vulnerability assessment and type-graph approaches which rely on an abstract model of the relations between attack types. Some research in to improving the efficiency of type-graph correlation has been carried out but this research has ignored the hypothesizing of missing alerts. Our work is to present a novel type-graph algorithm which unifies correlation and hypothesizing in to a single operation. Our experimental results indicate that the approach is extremely efficient in the face of intensive alerts and produces compact output graphs comparable to other techniques.

Modeling network intrusion detection alerts for correlation

Signature-based network intrusion-detection systems (NIDSs) often report a massive number of simple alerts of low-level security-related events. Many of these alerts are logically involved in a single multi-stage intrusion incident and a security officer often wants to analyze the complete incident instead of each individual simple alert. This paper proposes a well-structured model that abstracts the logical relation between the alerts in order to support automatic correlation of those alerts involved in the same intrusion. The basic building block of the model is a logical formula called a capability . We use capability to abstract consistently and precisely all levels of accesses obtained by the attacker in each step of a multistage intrusion. We then derive inference rules to define logical relations between different capabilities. Based on the model and the inference rules, we have developed several novel alert correlation algorithms and implemented a prototype alert correlator. The experimental results of the correlator using several intrusion datasets demonstrate that the approach is effective in both alert fusion and alert correlation and has the ability to correlate alerts of complex multistage intrusions. In several instances, the alert correlator successfully correlated more than two thousand Snort alerts involved in massive scanning incidents. It also helped us find two multistage intrusions that were missed in auditing by the security officers.

Visual correlation of network alerts

The VisAlert visual correlation tool facilitates situational awareness in complex network environments by providing a holistic view of network security to help detect malicious activities. Information visualization techniques and methods in many applications have effectively increased operators' situational awareness, letting them more effectively detect, diagnose, and treat anomalous conditions. Visualization elevates information comprehension by fostering rapid correlation and perceived associations. Our visualization technique integrates the information in log and alert files into an intuitive, flexible, extensible, and scalable visualization tool - VisAlert - that presents critical information concerning network activity in an integrated manner, increasing the user's situational awareness.

TRINETR: An architecture for collaborative intrusion detection and knowledge-based alert evaluation

Current reactive and standalone network security products are not capable of withstanding the onslaught of diversified network threats. As a result, a new security paradigm, where integrated security devices or systems collaborate closely to achieve enhanced protection and provide multi-layer defenses is emerging. In this paper, we present the design of a collaborative architecture for multiple intrusion detection systems to work together to detect real-time network intrusions. The detection is made more efficient and effective by using collaborative intelligent agents, relevant knowledge base and combination of multiple detection sensors. The architecture is composed of three parts: Collaborative Alert Aggregation, Knowledge-based Alert Evaluation and Alert Correlation. The architecture is aimed at reducing the alert overload by correlating results from multiple sensors to generate condensed views, reducing false positives by integrating network and host system information into the evaluation process and correlating events based on logical relations to generate global and synthesized alert report. The architecture is designed as a layer above intrusion detection for post-detection alert analysis and security actions. The first two parts of the architecture have been implemented and the implementation results are presented in this paper.

Comprehensive approach to intrusion detection alert correlation

Alert correlation is a process that analyzes the alerts produced by one or more intrusion detection systems and provides a more succinct and high-level view of occurring or attempted intrusions. Even though the correlation process is often presented as a single step, the analysis is actually carried out by a number of components, each of which has a specific goal. Unfortunately, most approaches to correlation concentrate on just a few components of the process, providing formalisms and techniques that address only specific correlation issues. This paper presents a general correlation model that includes a comprehensive set of components and a framework based on this model. A tool using the framework has been applied to a number of well-known intrusion detection data sets to identify how each component contributes to the overall goals of correlation. The results of these experiments show that the correlation components are effective in achieving alert reduction and abstraction. They also show that the effectiveness of a component depends heavily on the nature of the data set analyzed.

Détection d'intrusions : corrélation d'alertes

Current intrusion detection systems generate too many alerts. These alerts are imprecise and partial. Furthermore, they contain low level information. These alerts are therefore of limited interest for a human operator. Alert correlation is a promising technology to reduce the number of alerts, improve the diagnostic and provide a better vision of the security of the system in the case of an intrusion. This paper presents an overview of different alert correlation technologies and shows how these technologies can be applied to intrusion detection.

혼합형 침입 탐지 시스템에서 데이터 및 정책 전달 통신 모델과 성능 평가

침입 탐지 시스템에 대하여 많은 연구 노력들이 진행되고 있다. 그러나 침입 탐지 시스템의 모델과 성능 평가에 대한 작업은 거의 찾아 볼 수 없다. 본 논문에서는 지역적인 침입 탐지를 위한 에이전트들과 전역적인 침입 탐지를 위한 집중 데이터 분석 컴포넌트를 가지고 있는 다중 도메인 환경에서 혼합 침입 탐지를 위한 통신 프레임워크를 제안한다. 또한 전체적인 프레임워크에서 호스트 기반과 네트워크 기반 침입 탐지 시스템의 결합을 가정한다. 지역 도메인에서 경보와 로그 데이터 같은 정보 집합은 상위 레벨로 보고 된다. 계위의 루트에는 데이터 합동을 수행하는 전역 매니저가 있다. 전역 매니저는 침입 탐지 경보의 집합과 상호관련의 결과로 보안 정책을 하위 레벨로 전달하게 된다. 본 논문에서는 혼합 침입 탐지 시스템을 위한 통심 메커니즘을 모델링하고 데이터 및 정책 전달을 위한 전송 능력의 성능 평가를 위하여 OPNET 모델러를 이용한 시뮬레이터를 개발한다. 여러 가지 시나리오에 기반하여 통신 지연에 초점을 두고 모의실험 결과를 제시하고 비교한다. Much research efforts are being exerted for the study of intrusion detection system(IDS). However little work has been for the communication medels and performance eveluation of the IDS. Here we present a communication framework for doing hybrid intrusion detection in which agents are used for local intrusion detections with a centralized data anaysis componenta for a global intrusion detection at multiple domains environment. We also assume the combination of host-based and network-based intrusion detection systems in the oberall framework. From the local domain, a set of information such as alert, and / or log data are reported to the upper level. At the root of the hierarchy, there is a global manager where data coalescing is performed. The global manager delivers a security policy to its lower levels as the result of aggregation and correlation of intrusion detection alerts. In this paper, we model the communication mechanisms for the hybrid IDS and develop a simular using OPNET modeller for the performance evaluation of transmission capabillities for the delivery of data and policy. We present and compare simulation results based on several scenarios focuding on communication delay.

Eliminating noise from intrusion detection systems

Effective noise reduction for intrusion detection systems (IDS) is a continuous area of research. One of the techniques for eliminating unqualified IDS alerts is to correlate them with environmental intelligence about the network and systems. This article provides an overview of correlation requirements with a proposed architecture and solution for the correlation and classification of IDS alerts in real time. The implementation of the QuIDScor correlation engine was validated on a real-world network and demonstrated a significant reduction of false alerts.