Signature-based Intrusion Detection Systems Research Articles

A test of intrusion alert filtering based on network information

Intrusion detection systems continue to be a promising security technology. The arguably biggest problem with today's intrusion detection systems is the sheer number of alerts they produce for events that are regarded as benign or non-critical by system administrators. A plethora of more and less complex solutions has been proposed to filter the relevant i.e., correct alerts that signature-based intrusion detection sensors produce. This paper reports on a test performed to test a number of filtering alternatives that take advantage of information about static properties of the monitored computer network, such as vulnerabilities and exposure of ports and hosts. The results show that none of the filters are able to maintain a high recall portion of detected attacks while increasing the precision portion of relevant alerts. At most, precision increased from 1.4% to 2.9%, and this also resulted in a decrease in recall from 44% to 26%. Even when combined in an exploratory fashion, the filters fail to provide improved precision. It is concluded that filters based on static properties of the computer network do not result in clear improvements to alert lists produced by signature-based intrusion detection systems. Copyright © 2015 John Wiley & Sons, Ltd.

Signature-Based Method and Stream Data Mining Technique Performance Evaluation for Security and Intrusion Detection in Advanced Metering Infrastructures (AMI)

Advanced Metering Infrastructure is a system for collecting, measuring and analyzing power energy use which beside the mentioned parameters is responsible to transmit this information between elements of this network. One of the problems of processing in these networks and the most main installation duct is security in the network. Beside the precautionary operations used for achieving security in AMI, today the usage of intrusion detection systems (IDS) is paid attention as a second blocker defense against network security attacks. According to variations of Intrusion Detection techniques in network, many different designs for applying IDS and AMI have been suggested so far. In this paper we aim to evaluate the newest techniques of operation in abnormality-based IDS in Intrusion Detection and security achievement in AMI against signature-based IDS operation with the use of standard criteria in equal situations experimentally. According to the results from this research we can see that the signature-based methods are not useful in IDS systems applied in AMI network. The measurement on anomaly-based IDS which is using data mining techniques showed very hopeful results. However these methods need some improvements in order to achieve their best place.

Using response action with intelligent intrusion detection and prevention system against web application malware

Purpose – The purpose of this paper is to mitigate vulnerabilities in web applications, security detection and prevention are the most important mechanisms for security. However, most existing research focuses on how to prevent an attack at the web application layer, with less work dedicated to setting up a response action if a possible attack happened. Design/methodology/approach – A combination of a Signature-based Intrusion Detection System (SIDS) and an Anomaly-based Intrusion Detection System (AIDS), namely, the Intelligent Intrusion Detection and Prevention System (IIDPS). Findings – After evaluating the new system, a better result was generated in line with detection efficiency and the false alarm rate. This demonstrates the value of direct response action in an intrusion detection system. Research limitations/implications – Data limitation. Originality/value – The contributions of this paper are to first address the problem of web application vulnerabilities. Second, to propose a combination of an SIDS and an AIDS, namely, the IIDPS. Third, this paper presents a novel approach by connecting the IIDPS with a response action using fuzzy logic. Fourth, use the risk assessment to determine an appropriate response action against each attack event. Combining the system provides a better performance for the Intrusion Detection System, and makes the detection and prevention more effective.

An Organizational Role-based Extrusion Detection Model with Profile Migration

Intrusion detection and prevention systems play a crucial role in the overall information security implementation of today’s organizations. Traditionally, signature-based and anomaly-based detections have been the two main methods of detection and prevention techniques. Signature-based intrusion detection systems are excellent in detection and performance, but they are vulnerable to unknown threats like zero-day attacks. Extensive research have been conducted on anomaly detection and prevention based on users’ behavior profiling. However, as insider attacks increase, it has become equally important to monitor and analyze extrusion attempts. Behavior-based profile creation has a promising future in extrusion monitoring. However, profiling individual behavior has its limitations in that it tends to incorporate unintended behavior into the normal profile. In this study, user's organizational role has been integrated into profile creation further reducing number of false positives. A prototype of the model is tested with three users belonging to three different roles. A profile migration scheme is proposed to import user profiles at various login location.

Enhancing the performance of signature-based network intrusion detection systems: an engineering approach

Signature-based network intrusion detection systems (NIDSs) have been popularly implemented in different organisations, with the purpose of defending against various attacks. However, it is identified that these systems suffer from three major issues in practical applications such as overload packets, expensive signature matching and massive false alarms, which would significantly decrease the effectiveness of these systems. In this paper, an adaptive framework is proposed to improve the overall performance of a signature-based NIDS such as Snort regarding the aforementioned issues. This framework is further implemented in an engineering way, in which a trust-based packet filter with an exclusive signature matching scheme, and an intelligent machine learning-based false alarm filter aiming to reduce target packets, improve the process of signature matching and decrease the number of false alarms are constructed, respectively. In the evaluation, the experimental results on a well-known benchmark and a real...

A roadmap towards improving managed security services from a privacy perspective

This paper proposes a roadmap for how privacy leakages from outsourced managed security services using intrusion detection systems can be controlled. The paper first analyses the risk of leaking private or confidential information from signature-based intrusion detection systems. It then discusses how the situation can be improved by developing adequate privacy enforcement methods and privacy leakage metrics in order to control and reduce the leakage of private and confidential information over time. Such metrics should allow for quantifying how much information that is leaking, where these information leakages are, as well as showing what these leakages mean. This includes adding enforcement mechanisms ensuring that operation on sensitive information is transparent and auditable. The data controller or external quality assurance organisations can then verify or certify that the security operation operates in a privacy friendly manner. The roadmap furthermore outlines how privacy-enhanced intrusion detection systems should be implemented by initially providing privacy-enhanced alarm handling and then gradually extending support for privacy enhancing operation to other areas like digital forensics, exchange of threat information and big data analytics based attack detection.

Network Intrusion Detection using Layered Approach and Hidden Markov Model

intrusion detection systems uses either anomaly based or signature based technique. Both of these techniques have some problems. In anomaly based intrusion detection, the strategy is to suspect an unusual activity and thereby to continue further investigation. This approach is particularly effective against novel attacks. Signature based intrusion detection system detects known attacks timely and efficiently. For this approach, it is important to know the attack. The proposed system introduces a hybrid of anomaly based and signature based technique. The proposed system uses layered approach to get the results faster. Each layer in the layered approach is independent to detect and block an attack. Four different layers Probe, U2R, R2L and DOS are assigned with different features. The proposed hybrid technique with Hidden Markov Model can give better results compared to signature based and anomaly based intrusion detection techniques alone.

Randomized Anagram revisited

When compared to signature-based Intrusion Detection Systems (IDS), anomaly detectors present the potential advantage of detecting previously unseen attacks, which makes them an attractive solution against zero-day exploits and other attacks for which a signature is unavailable. Most anomaly detectors rely on machine learning algorithms to derive a model of normality that is later used to detect suspicious events. Such algorithms, however, are generally susceptible to evasion by means of carefully constructed attacks that are not recognized as anomalous. Different strategies to thwart evasion have been proposed over the last years, including the use of randomization to make somewhat uncertain how each packet will be processed. In this paper we analyze the strength of the randomization strategy suggested for Anagram, a well-known anomaly detector based on n-gram models. We show that an adversary who can interact with the system for a short period of time with inputs of his choosing will be able to recover the secret mask used to process packets. We describe and discuss an efficient algorithm to do this and report our experiences with a prototype implementation. Furthermore, we show that the specific form of randomization suggested for Anagram is a double-edged sword, as knowledge of the mask makes evasion easier than in the non-randomized case. We finally discuss a simple countermeasure to prevent our attacks.

Cloud security auditing based on behavioural modelling

Multi-tenancy is one of the most attractive features of cloud computing, which provides significant benefits to both clients and service providers by supporting elastic, efficient and on-demand resource provisioning and allocation. Multi-tenancy also introduces additional security auditing opportunities. Security auditing can be consolidated and offloaded onto a dedicated and well-protected service. The timely prevention of intrusive behaviour and malicious processes using signature-based intrusion detection technologies or system call level anomaly analysis is a very challenging task due to a high rate of false alarms. In this work, a behavioural modelling scheme is proposed to audit the behaviours of client virtual machine and detect suspicious processes on the level of functionality. The proposed scheme can be used as a community security auditing service. The scheme can also be used by cloud providers to offer automatic identification and auditing of the tenant’s services. Our preliminary results have...

Adaptive non-critical alarm reduction using hash-based contextual signatures in intrusion detection

Signature-based intrusion detection systems (IDSs) have been widely deployed in network environments aiming to defend against different kinds of attacks. However, a large number of alarms, especially noncritical alarms could be generated during the detection, which can greatly lower the effectiveness of detection and increase the difficulty in analyzing the generated IDS alarms. The main reason is that the detection capability of a signature-based IDS heavily depends on its signatures, whereas current IDS signatures are short of information related to actual deployment (i.e., lacking of contextual information). In addition, the traditional signature matching is a key limiting factor for IDSs in which the processing burden is at least linear to the size of an input string. To mitigate these issues, in this paper, we propose a novel scheme of hash-based contextual signatures that combines the original intrusion detection signatures with contextual information and hash functions. By using hash functions, our scheme can be used to construct an adaptive hash-based non-critical alarm filter which can further improve the performance of existing contextual signatures in filtering out non-critical alarms. Some examples of contextual information matching are also provided. In the evaluation, we discuss how to choose appropriate hash functions and investigate the performance upon implementation of the scheme with a real dataset and in a real network environment. The experimental results are positive and indicate that our scheme is encouraging and effective in filtering out non-critical alarms.

Automatic attack signature generation systems: A review

Signature-based intrusion detection systems provide solutions to counter the increasing number of attacks on network resources. But this is not helpful for novel attacks whose signatures aren't available. Automated signature generation systems can work proactively to detect these attacks in real time and generate signatures of new attacks. This article analyzes the latest developments, including Honeycyber, Hancock, Arbor, Auto-Sign, Argos, Hamsa, F-Sign, and a hybrid honeyfarm-based defense system, comparing these systems on the basis of their ability to detect novel attacks, signature generation method, suitability for multiple instances of worms, type of signature generated, attacks and worms covered, false alarm rates, and relative strengths and weaknesses.

THE USE OF RANDOM FOREST CLASSIFICATION AND K-MEANS CLUSTERING ALGORITHM FOR DETECTING TIME STAMPED SIGNATURES IN THE ACTIVE NETWORKS

In day to day information security infrastructure, intrusion detection is indispensible. Signature bas ed intrusion detection system mechanisms are often ava ilable in detecting many types of attacks. But this mechanism alone is not sufficient in many cases. An other intrusion detection method viz K-means is employed for clustering and classifying the unlabel led data. IDS is a special embedded device or relie d software package which process of monitoring the events occurring in a computer system or network (WLAN (Wi-Fi, Wimax)) and LAN ((Ethernet, FDDI, ADSL, Token ring) based) and analysing them for sign of possible incident which are violations or f orthcoming threats of violations of computer securi ty policies or standard security policies (i.e., DMA a cts). We proposed a new methodology for detecting intrusions by means of clustering and classificatio n algorithms. There we used correlation clustering and Kmeans clustering algorithm for clustering and rando m forest algorithm for classification. This type of extension establishes a layer which refines the esc alated alerts using signature-based correlation. In this study, signature based intrusion detection system w ith optimised algorithm for better prediction of intrusions has been addressed. Results are presente d and discussed.

(WHASG) Automatic SNORT Signatures Generation by using Honeypot

An Intrusion detection system (IDS) is an important network security component that is used to monitor network traffic and detect attack attempts. A signature based intrusion detection system relies on a set of predefined signatures to detect an attack. Due to zero-day attacks (i.e. new unknown attacks) conventional IDS will not be able to detect these new attacks until the signatures are updated. Writing efficient new signatures to update the IDS signature database requires that the attack is first detected then studied and analyzed. These new rules should be general enough to include any modification of the attack pattern and specific so that normal traffic remains unblocked. Writing these signatures manually requires significant effort, time and knowledge to work properly. In this paper, a web based honeypot is used to generate SNORT intrusion detection system signatures (Rules) for HTTP traffic automatically. These new rules are integrated into the IDS signatures data base. We then verify the efficiency of the modified rules and show that the new rules are able to detect and block these attacks.

An Improved Hybrid Intrusion Detection System in Cloud Computing

ssecurity is a major concern. Cloud computing and Intrusion Detection and Prevention Systems are one such measure to mitigate these attacks. Different researchers have proposed different IDSs time to time some of these IDS's combine features of two or more IDSs which are called as Hybrid Intrusion Detection Systems. Most of the researchers combine the features of Signature based detection methodology and Anomaly based detection methodology. For a signature based IDS if an attacker attacks slowly and organized, the attack may go undetected through the IDS, as signatures include factors which are based on duration of the events and the actions of attacker do not match. Sometimes, for an unknown attack there is no signature updated or an attacker attack in the mean time when the database is updating. Thus, signature-based IDS fail to detect unknown attacks. Anomaly based IDS suffer from many false-positive readings. Thus there is a need to hybridize those IDS which can overcome the shortcomings of each other. In this paper we proposed a new approach to IDS (Intrusion Detection System) which is more efficient than the traditional IDS (Intrusion Detection System). The IDS is based on Honeypot Technology and Anomaly based Detection Methodology. We have designed Architecture for the IDS in a packet tracer and then implemented it in real time. We have discussed experimental results performed both the Honeypot and Anomaly based IDS have some shortcomings but if we hybridized these two technologies, the newly proposed HIDS is capable enough to overcome these shortcomings with much enhanced performance. In this paper, we present a modified Hybrid Intrusion Detection System (HIDS) that combines the positive features of two different detection methodologies - Honeypot methodology and anomaly based intrusion detection methodology. In the experiment we run both the Intrusion Detection System individually first and then together and record the data from time to time. From the data we can conclude that the resulting IDS is much better in detecting intrusions from the existing IDSs.

A Pareto-based multi-objective evolutionary algorithm for automatic rule generation in network intrusion detection systems

Attacks against computer systems are becoming more complex, making it necessary to continually improve the security systems, such as intrusion detection systems which provide security for computer systems by distinguishing between hostile and non-hostile activity. Intrusion detection systems are usually classified into two main categories according to whether they are based on misuse (signature-based) detection or on anomaly detection. With the aim of minimizing the number of wrong decisions, a new Pareto-based multi-objective evolutionary algorithm is used to optimize the automatic rule generation of a signature-based intrusion detection system (IDS). This optimizer, included within a network IDS, has been evaluated using a benchmark dataset and real traffic of a Spanish university. The results obtained in this real application show the advantages of using this multi-objective approach.

Data Stream Subspace Clustering for Anomalous Network Packet Detection

As the Internet offers increased connectivity between human beings, it has fallen prey to malicious users who exploit its resources to gain illegal access to critical information. In an effort to protect computer networks from external attacks, two common types of Intrusion Detection Systems (IDSs) are often deployed. The first type is signature-based IDSs which can detect intrusions efficiently by scanning network packets and comparing them with human-generated signatures describing previously-observed attacks. The second type is anomaly-based IDSs able to detect new attacks through modeling normal network traffic without the need for a human expert. Despite this advantage, anomaly-based IDSs are limited by a high false-alarm rate and difficulty detecting network attacks attempting to blend in with normal traffic. In this study, we propose a StreamPreDeCon anomaly-based IDS. StreamPreDeCon is an extension of the preference subspace clustering algorithm PreDeCon designed to resolve some of the challenges associated with anomalous packet detection. Using network packets extracted from the first week of the DARPA '99 intrusion detection evaluation dataset combined with Generic Http, Shellcode and CLET attacks, our IDS achieved 94.4% sensitivity and 0.726% false positives in a best case scenario. To measure the overall effectiveness of the IDS, the average sensitivity and false positive rates were calculated for both the maximum sensitivity and the minimum false positive rate. With the maximum sensitivity, the IDS had 80% sensitivity and 9% false positives on average. The IDS also averaged 63% sensitivity with a 0.4% false positive rate when the minimal number of false positives is needed. These rates are an improvement on results found in a previous study as the sensitivity rate in general increased while the false positive rate decreased.

Evasion Techniques: Sneaking through Your Intrusion Detection/Prevention Systems

Detecting attacks disguised by evasion techniques is a challenge for signature-based Intrusion Detection Systems (IDSs) and Intrusion Prevention Systems (IPSs). This study examines five common evasion techniques to determine their ability to evade recent systems. The denial-of-service (DoS) attack attempts to disable a system by exhausting its resources. Packet splitting tries to chop data into small packets, so that a system may not completely reassemble the packets for signature matching. Duplicate insertion can mislead a system if the system and the target host discard different TCP/IP packets with a duplicate offset or sequence. Payload mutation fools a system with a mutative payload. Shellcode mutation transforms an attacker's shellcode to escape signature detection. This study assesses the effectiveness of these techniques on three recent signature-based systems, and among them, explains why Snort can be evaded. The results indicate that duplicate insertion becomes less effective on recent systems, but packet splitting, payload mutation and shellcode mutation can be still effective against them.

Survey on Incremental Approaches for Network Anomaly Detection

As the communication industry has connected distant corners of the globe using advances in network technology, intruders or attackers have also increased attacks on networking infrastructure commensurately. System administrators can attempt to prevent such attacks using intrusion detection tools and systems. There are many commercially available signature-based Intrusion Detection Systems (IDSs). However, most IDSs lack the capability to detect novel or previously unknown attacks. A special type of IDSs, called Anomaly Detection Systems, develop models based on normal system or network behavior, with the goal of detecting both known and unknown attacks. Anomaly detection systems face many problems including high rate of false alarm, ability to work in online mode, and scalability. This paper presents a selective survey of incremental approaches for detecting anomaly in normal system or network traffic. The technological trends, open problems, and challenges over anomaly detection using incremental approach are also discussed.

A fast pattern matching algorithm with multi-byte search unit for high-speed network security

A signature-based intrusion detection system identifies intrusions by comparing the data traffic with known signature patterns. In this process, matching of packet strings against signature patterns is the most time-consuming step and dominates the overall system performance. Many signature-based network intrusion detection systems (NIDS), e.g., the Snort, employ one or multiple pattern matching algorithms to detect multiple attack types. So far, many pattern matching algorithms have been proposed. Most of them use single-byte standard unit for search, while a few algorithms such as the Modified Wu–Manber (MWM) algorithm use typically two-byte unit, which guarantees better performance than others even as the number of different signatures increases. Among those algorithms, the MWM algorithm has been known as the fastest pattern matching algorithm when the patterns in a rule set rarely appear in packets. However, the matching time of the MWM algorithm increases as the length of the shortest pattern in a signature group decreases. In this paper, by extending the length of the shortest pattern, we minimize the pattern matching time of the algorithm which uses multi-byte unit. We propose a new pattern matching algorithm called the L +1-MWM algorithm for multi-pattern matching. The proposed algorithm minimizes the performance degradation that is originated from the dependency on the length of the shortest pattern. We show that the L +1-MWM algorithm improves the performance of the MWM algorithm by as much as 20% in average under various lengths of shortest patterns and normal traffic conditions. Moreover, when the length of the shortest pattern in a rule set is less than 5, the L +1-MWM algorithm shows 38.87% enhancement in average. We also conduct experiments on a real campus network and show that 12.48% enhancement is obtained in average. In addition, it is shown that the L +1-MWM algorithm provides a better performance than the MWM algorithm by as much as 25% in average under various numbers of signatures and normal traffic conditions, and 20.12% enhancement in average with real on-line traffic.

Anomalous Network Packet Detection Using Data Stream Mining

In recent years, significant research has been devoted to the development of Intrusion Detection Systems (IDS) able to detect anomalous computer network traffic indicative of malicious activity. While signature-based IDS have proven effective in discovering known attacks, anomaly-based IDS hold the even greater promise of being able to automatically detect previously undocumented threats. Traditional IDS are generally trained in batch mode, and therefore cannot adapt to evolving network data streams in real time. To resolve this limitation, data stream mining techniques can be utilized to create a new type of IDS able to dynamically model a stream of network traffic. In this paper, we present two methods for anomalous network packet detection based on the data stream mining paradigm. The first of these is an adapted version of the DenStream algorithm for stream clustering specifically tailored to evaluate network traffic. In this algorithm, individual packets are treated as points and are flagged as normal or abnormal based on their belonging to either normal or outlier clusters. The second algorithm utilizes a histogram to create a model of the evolving network traffic to which incoming traffic can be compared using Pearson correlation. Both of these algorithms were tested using the first week of data from the DARPA ’99 dataset with Generic HTTP, Shell-code and Polymorphic attacks inserted. We were able to achieve reasonably high detection rates with moderately low false positive percentages for different types of attacks, though detection rates varied between the two algorithms. Overall, the histogram-based detection algorithm achieved slightly superior results, but required more parameters than the clustering-based algorithm. As a result of its fewer parameter requirements, the clustering approach can be more easily generalized to different types of network traffic streams.