Signature-based Intrusion Detection Systems Research Articles

Efficient Intrusion Detection System for SDN Orchestrated Internet of Things

Internet of Things (IoT) can simply be defined as an extension of the current Internet system. It extends the human to human interconnection and intercommunication scenario of the Internet by including things, to bring anytime, anywhere, and anything communication. A discipline in networking evolving in parallel with IoT is Software Defined Networking (SDN). It is an important technology that is aimed to solve the different problems existing in the traditional network systems. It provides a new convenient home to address the different challenges existing in different network-based systems including IoT. One important security challenge prevailing in such SDN-based IoT (SDIoT) systems is guarantying service availability. The ever-increasing denial of service (DoS) attacks are responsible for such service denials. A centralized signature-based intrusion detection system (IDS) is proposed and developed in this work. Random Forest (RF) classifier is used for training the model. A very popular and recent benchmark dataset, CICIDS2017, has been used for training and validating the machine learning (ML) models. An accuracy result of 99.968% has been achieved by using only 12 features on Wednesday’s release of the dataset. This result is higher than the achieved accuracy results of related works considering the original CICIDS2017 dataset. A maximum cross-validated accuracy result of 99.713% has been achieved on the same release of the dataset. These developed models meet the basic requirement of a supervised IDS system developed for smart environments and can effectively be used in different IoT service scenarios.

Exploiting the Outcome of Outlier Detection for Novel Attack Pattern Recognition on Streaming Data

Future-oriented networking infrastructures are characterized by highly dynamic Streaming Data (SD) whose volume, speed and number of dimensions increased significantly over the past couple of years, energized by trends such as Software-Defined Networking or Artificial Intelligence. As an essential core component of network security, Intrusion Detection Systems (IDS) help to uncover malicious activity. In particular, consecutively applied alert correlation methods can aid in mining attack patterns based on the alerts generated by IDS. However, most of the existing methods lack the functionality to deal with SD data affected by the phenomenon called concept drift and are mainly designed to operate on the output from signature-based IDS. Although unsupervised Outlier Detection (OD) methods have the ability to detect yet unknown attacks, most of the alert correlation methods cannot handle the outcome of such anomaly-based IDS. In this paper, we introduce a novel framework called Streaming Outlier Analysis and Attack Pattern Recognition, denoted as SOAAPR, which is able to process the output of various online unsupervised OD methods in a streaming fashion to extract information about novel attack patterns. Three different privacy-preserving, fingerprint-like signatures are computed from the clustered set of correlated alerts by SOAAPR, which characterizes and represents the potential attack scenarios with respect to their communication relations, their manifestation in the data’s features and their temporal behavior. Beyond the recognition of known attacks, comparing derived signatures, they can be leveraged to find similarities between yet unknown and novel attack patterns. The evaluation, which is split into two parts, takes advantage of attack scenarios from the widely-used and popular CICIDS2017 and CSE-CIC-IDS2018 datasets. Firstly, the streaming alert correlation capability is evaluated on CICIDS2017 and compared to a state-of-the-art offline algorithm, called Graph-based Alert Correlation (GAC), which has the potential to deal with the outcome of anomaly-based IDS. Secondly, the three types of signatures are computed from attack scenarios in the datasets and compared to each other. The discussion of results, on the one hand, shows that SOAAPR can compete with GAC in terms of alert correlation capability leveraging four different metrics and outperforms it significantly in terms of processing time by an average factor of 70 in 11 attack scenarios. On the other hand, in most cases, all three types of signatures seem to reliably characterize attack scenarios such that similar ones are grouped together, with up to 99.05% similarity between the FTP and SSH Patator attack.

COMPARATIVE EVALUATION OF MACHINE LEARNING METHODS FOR NETWORK INTRUSION DETECTION SYSTEM

Cyber security is becoming more sophisticated, and as a result, there is an increasing challenge to accurately detect intrusions. Lack of intrusion prevention can degrade the credibility of security services, namely data confidentiality, integrity and availability. Many intrusion detection methods have been suggested in the literature to address threats to computer security, which can be broadly classified into signature-based intrusion detection (SIDS) and anomaly-based intrusion detection systems. (AIDS). This research presents the contemporary taxonomy of IDS, a comprehensive review of important recent work, and an overview of commonly used datasets for assessment purposes. It also presents detail analysis of different machine learning approach for intrusion detection.

Application of Bioinformatics Algorithms for 3RO\PRUSKLF Cyberattacks Detection

The functionality of any system can be represented as a set of commands that lead to a change in the state of the system. The intrusion detection problem for signature-based intrusion detection systems is equivalent to matching the sequences of operational commands executed by the protected system to known attack signatures. Various mutations in attack vectors (including replacing commands with equivalent ones, rearranging the commands and their blocks, adding garbage and empty commands into the sequence) reduce the effectiveness and accuracy of the intrusion detection. The article analyzes the existing solutions in the field of bioinformatics and considers their applicability for solving the problem of identifying polymorphic attacks by signature-based intrusion detection systems. A new approach to the detection of polymorphic attacks based on the suffix tree technology applied in the assembly and verification of the similarity of genomic sequences is discussed. The use of bioinformatics technology allows us to achieve high accuracy of intrusion detection at the level of modern intrusion detection systems (more than 0.90), while surpassing them in terms of cost-effectiveness of storage resources, speed and readiness to changes in attack vectors. To improve the accuracy indicators, a number of modifications of the developed algorithm have been carried out, as a result of which the accuracy of detecting attacks increased by up to 0.95 with the level of mutations in the sequence up to 10%. The developed approach can be used for intrusion detection both in conventional computer networks and in modern reconfigurable network infrastructures with limited resources (Internet of Things, networks of cyber-physical objects, wireless sensor networks).

Designing consensus algorithm for collaborative signature-based intrusion detection system

<span>Signature-based collaborative intrusion detection system (CIDS) is highly depends on the reliability of nodes to provide IDS attack signatures. Each node in the network is responsible to provide new attack signature to be shared with other node. There are two problems exist in CIDS highlighted in this paper, first is to provide data consistency and second is to maintain trust among the nodes while sharing the attack signatures. Recently, researcher find that blockchain has a great potential to solve those problems. Consensus algorithm in blockchain is able to increase trusts among the node and allows data to be inserted from a single source of truth. In this paper, we are investigating three blockchain consensus algorithms: proof of work (PoW), proof of stake (PoS), and hybrid PoW-PoS chain-based consensus algorithm which are possibly to be implemented in CIDS. Finally, we design an extension of hybrid PoW-PoS chain-based consensus algorithm to fulfill the requirement. This extension we name it as proof of attack signature (PoAS).</span>

AS-IDS: Anomaly and Signature Based IDS for the Internet of Things

The Internet of Things (IoT) is a massively extensive environment that can manage many diverse applications. Security is critical due to potential malicious threats and the diversity of the connectivity. Devices can protect themselves and detect threats with the Intrusion Detection System (IDS). IDS typically uses one of two approaches: anomaly-based or signature-based. This paper proposes a model (known as “AS-IDS”) that combines these two approaches to detect known and unknown attacks in IoT networks. The proposed model has three phases: traffic filtering, preprocessing and the hybrid IDS. In the first phase, the arrival traffic is filtered at the IoT gateway by matching packet features, after which the preprocessing phase applies a Target Encoder, Z-score and Discrete Hessian Eigenmap (DHE) to encode, normalize and eliminate redundancy, respectively. In the final phase, the hybrid IDS integrates signatures and anomalies. The signature-based IDS subsystem investigates packets with Lightweight Neural Network (LightNet), which uses Human Mental Search (HMS) for traffic clustering in the hidden layer and Boyer Moore is used to search for a particular signature in the output layer that is accelerated by using the Generalized Suffix Tree (GST) algorithm and by matching the signatures it classifies the attacks as intruder, normal or unknown. The anomaly-based IDS subsystem employs Deep Q-learning to identify unknown attacks, and uses Signal to Noise Ratio (SNR) and bandwidth to classify the attacks into five classes: Denial of Service (DoS), Probe, User-to-Root (U2R), Remote-to-Local (R2L), and normal traffic. Detected packets are then generated with new signatures, using the Position Aware Distribution Signature (PADS) algorithm. The proposed AS-IDS is implemented in real-time traffic with the NSL-KDD dataset, and the results are evaluated in terms of Detection Rate (DR), False Alarm Rate (FAR), Specificity, F-measure and computation time.

Acceleration of Intrusion Detection in Encrypted Network Traffic Using Heterogeneous Hardware.

More than 75% of Internet traffic is now encrypted, and this percentage is constantly increasing. The majority of communications are secured using common encryption protocols such as SSL/TLS and IPsec to ensure security and protect the privacy of Internet users. However, encryption can be exploited to hide malicious activities, camouflaged into normal network traffic. Traditionally, network traffic inspection is based on techniques like deep packet inspection (DPI). Common applications for DPI include but are not limited to firewalls, intrusion detection and prevention systems, L7 filtering, and packet forwarding. With the widespread adoption of network encryption though, DPI tools that rely on packet payload content are becoming less effective, demanding the development of more sophisticated techniques in order to adapt to current network encryption trends. In this work, we present HeaderHunter, a fast signature-based intrusion detection system even for encrypted network traffic. We generate signatures using only network packet metadata extracted from packet headers. In addition, we examine the processing acceleration of the intrusion detection engine using different heterogeneous hardware architectures.

A signature-based intrusion detection system using a hybrid Decision Table and Naive Bayes

A signature-based intrusion detection system using a hybrid Decision Table and Naive Bayes

Attack Resilient Trust and Signature-based Intrusion Detection Systems

Wireless sensor networks have been widely applied in many areas due to their unique characteristics. These have exposed them to different types of active and passive attacks. In the literature, several solutions have been proposed to mitigate these attacks. Most of the proposed solutions are too complex to be implemented in wireless sensor networks considering the resource-constraint of sensor nodes. In this work, we proposed a hierarchical trust mechanism based on clustering approach to detect and prevent denial of service attacks in wireless sensor networks. The approach was validated through simulation using Network Simulator (NS2). The following metrics were used to evaluation the proposed scheme: packet delivery ratio, network lifetime, routing delay, overhead, and number of nodes. The proposed approach is capable of detecting compromised sensor nodes vulnerable to a denial of service attacks. Moreover, it is able to detect all sensed data that have been compromised during transmission to the base station. The results show that our method can effectively detect and defend against denial of service attacks in sensor wireless sensor networks.

Network Anomaly Detection Using Memory-Augmented Deep Autoencoder

In recent years, attacks on network environments continue to rapidly advance and are increasingly intelligent. Accordingly, it is evident that there are limitations in existing signature-based intrusion detection systems. In particular, for novel attacks such as Advanced Persistent Threat (APT), signature patterns have problems with poor generalization performance. Furthermore, in a network environment, attack samples are rarely collected compared to normal samples, creating the problem of imbalanced data. Anomaly detection using an autoencoder has been widely studied in this environment, and learning is through semi-supervised learning methods to overcome these problems. This approach is based on the assumption that reconstruction errors for samples that are not used for training will be large, but an autoencoder is often over-generalized and this assumption is often broken. In this paper, we propose a network intrusion detection method using a memory-augmented deep auto-encoder (MemAE) that can solve the over-generalization problem of autoencoders. The MemAE model is trained to reconstruct the input of an abnormal sample that is close to a normal sample, which solves the generalization problem for such abnormal samples. Experiments were conducted on the NSL-KDD, UNSW-NB15, and CICIDS 2017 datasets, and it was confirmed that the proposed method is better than other one-class models.

Decepticon: a Theoretical Framework to Counter Advanced Persistent Threats

Deception has been proposed in the literature as an effective defense mechanism to address Advanced Persistent Threats (APT). However, administering deception in a cost-effective manner requires a good understanding of the attack landscape. The attacks mounted by APT groups are highly diverse and sophisticated in nature and can render traditional signature based intrusion detection systems useless. This necessitates the development of behavior oriented defense mechanisms. In this paper, we develop Decepticon (Deception-based countermeasure), a Hidden Markov Model based framework where the indicators of compromise (IoC) are used as the observable features to aid in detection. This theoretical framework also includes several models to represent the spread of APTs in a computer system. The presented framework can be used to select an appropriate deception script when faced with APTs or other similar malware and trigger an appropriate defensive response. The effectiveness of the models in a networked system is illustrated by considering a real APT type ransomware.

INTRUSION DETECTION IN COMPUTER NETWORKS USING LATENT SPACE REPRESENTATION AND MACHINE LEARNING

Anomaly detection (AD) identifies samples that are not related to the overall distribution in the feature space. This problem has a long history of research through diverse methods, including statistical and modern Deep Neural Networks (DNN) methods. Non-trivial tasks such as covering ambiguous user actions and the complexity of standard algorithms challenged researchers. This article discusses the results of introducing an intrusion detection system using a machine learning (ML) approach. We compared these results with the characteristics of the most common existing rule-based Snort system. Signature Based Intrusion Detection System (SBIDS) has critical limitations well observed in a large number of previous studies. The crucial disadvantage is the limited variety of the same attack type due to the predetermination of all the rules. DNN solves this problem with long short-term memory (LSTM). However, requiring the amount of data and resources for training, this solution is not suitable for a real-world system. This necessitated a compromise solution based on DNN and latent space techniques.

Anomaly detection optimization using big data and deep learning to reduce false-positive

Anomaly-based Intrusion Detection System (IDS) has been a hot research topic because of its ability to detect new threats rather than only memorized signatures threats of signature-based IDS. Especially after the availability of advanced technologies that increase the number of hacking tools and increase the risk impact of an attack. The problem of any anomaly-based model is its high false-positive rate. The high false-positive rate is the reason why anomaly IDS is not commonly applied in practice. Because anomaly-based models classify an unseen pattern as a threat where it may be normal but not included in the training dataset. This type of problem is called overfitting where the model is not able to generalize. Optimizing Anomaly-based models by having a big training dataset that includes all possible normal cases may be an optimal solution but could not be applied in practice. Although we can increase the number of training samples to include much more normal cases, still we need a model that has more ability to generalize. In this research paper, we propose applying deep model instead of traditional models because it has more ability to generalize. Thus, we will obtain less false-positive by using big data and deep model. We made a comparison between machine learning and deep learning algorithms in the optimization of anomaly-based IDS by decreasing the false-positive rate. We did an experiment on the NSL-KDD benchmark and compared our results with one of the best used classifiers in traditional learning in IDS optimization. The experiment shows 10% lower false-positive by using deep learning instead of traditional learning.

Diverse Methods for Signature based Intrusion Detection Schemes Adopted

Intrusion Detection Systems (IDS) is used as a tool to detect intrusions on IT networks, providing support in network monitoring to identify and avoid possible attacks. Most such approaches adopt Signature-based methods for detecting attacks which include matching the input event to predefined database signatures. Signature based intrusion detection acts as an adaptable device security safeguard technology. This paper discusses various Signature-based Intrusion Detection Systems and their advantages; given a set of signatures and basic patterns that estimate the relative importance of each intrusion detection system feature, system administrators may help identify cyber-attacks and threats to the network and Computer system. Eighty percent of incidents can be easily and promptly detected using signature-based detection methods if used as a precautionary phase for vulnerability detection and twenty percent rest by anomaly-based intrusion detection system that involves comparing definitions of normal activity or event behavior with observed events in identifying the significant deviations and deciding the traffic to flag.

Towards blockchain-enabled single character frequency-based exclusive signature matching in IoT-assisted smart cities

With the increasing viability of Internet of Things (IoT), more devices are expected to be connected in a smart city environment. It can provide many benefits for people’s daily life, but is also susceptible to many security threats in practice. Intrusion detection systems (IDSs), especially signature-based IDSs, are one of the most commonly adopted security mechanisms to safeguard various network environments like IoT-assisted smart city against cyber attacks. The process of signature matching is a key limiting factor for a signature-based IDS, and the exclusive signature matching (ESM) was designed based on the observation that most network packets would not match any IDS signatures. However, exclusive signature matching like the single character frequency-based ESM may be vulnerable to some attacks in a hostile environment. To mitigate this issue, in this work, we propose a blockchain-enabled single character frequency-based ESM, which can build a verifiable database of malicious payloads via blockchains. In the evaluation, we investigate the performance of our approach under flooding and character padding attacks in both a simulated and a real IoT network environment. The results demonstrate the effectiveness of our approach in enhancing the robustness of single character frequency-based ESM against malicious traffic.

Intrusion Detection Systems for Mitigating SQL Injection Attacks

Databases are widely used by organizations to store business-critical information, which makes them one of the most attractive targets for security attacks. SQL Injection is the most common attack to webpages with dynamic content. To mitigate it, organizations use Intrusion Detection Systems (IDS) as part of the security infrastructure, to detect this type of attack. However, the authors observe a gap between the comprehensive state-of-the-art in detecting SQL Injection attacks and the state-of-practice regarding existing tools capable of detecting such attacks. The majority of IDS implementations provide little or no protection against SQL Injection attacks, with exceptions like the tools Bro and ModSecurity. In this article, the authors compare these tools using the CSIC dataset in order to examine the state-of-practice in database protection from SQL Injection attacks, identifying the main characteristics and implementation details needed for IDSs to successfully detect such attacks. The experiments indicate that signature-based IDS provide the greatest coverage against SQL Injection.

Analysis of hidden Markov model learning algorithms for the detection and prediction of multi-stage network attacks

Hidden Markov Models have been extensively used for determining computer systems under a Multi-Stage Network Attack (MSA), however, acquisition of optimal model training parameters remains a formidable challenge. This paper critically analyses the detection and prediction accuracy of a wide range of training and initialisation algorithms including the expectation–maximisation, spectral, Baum–Welch, differential evolution, K-means, and segmental K-means. The performance of these algorithms has been evaluated, both individually and in a hybrid approach, for detecting all the states and current state, and predicting the next state (NS), and the next observation (NO) of a given alert observation sequence. For generating this alert sequence, the Snort signature-based intrusion detection system was utilised, using either bespoke or default rules, to raise alerts while examining the DARPA 2000 MSA dataset. The investigation also sheds further light on alternative approaches for forecasting the possible NS and NO in an MSA campaign, as well as, the impact of window size on the prediction performance for all analysed techniques. The results and discussion emphasise on the appropriateness of various techniques for the prediction of NS and NO. Furthermore, NO prediction accuracy has indicated a performance increase of up to 44.95% in the proposed hybrid approaches.

On High-Speed Flow-Based Intrusion Detection Using Snort-Compatible Signatures

Signature-based Network Intrusion Detection Systems (NIDS) have become state-of-the-art in modern network security solutions. However, most systems are not designed for modern high-speed network links. In the field of network monitoring, an alternative solution has become the choice for such high-speed networks. Flow-monitoring, typically based on the Internet Protocol Flow Information Export (IPFIX) standard, now goes well beyond collecting statistical information about network connections. Current solutions are even able to include selected parts of the payload in these Flows to be used in conjunction with NIDS. Recently, we extended this concept to application layer HTTP Flows. We now present our improved version of the IPFIX-based Signature-based Intrusion Detection System (FIXIDS). <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Fixids</small> makes use of HTTP intrusion detection signatures from the popular Snort system and applies them to incoming IPFIX-conforming HTTP Flows. Our evaluation shows that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Fixids</small> can deal with four times higher network data rates without drops compared to Snort, while maintaining the same event detection rate. Furthermore, a substantial part of the data traffic can be outsourced to <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Fixids</small> so that Snort can be relieved of a significant portion of rules and traffic. This increases both the detection rate and the data rate the overall security appliance can handle.

An intelligent flow-based and signature-based IDS for SDNs using ensemble feature selection and a multi-layer machine learning-based classifier

Software-defined networking is a new paradigm that overcomes problems associated with traditional network architecture by separating the control logic from data plane devices. It also enhances performance by providing a highly-programmable interface that adapts to dynamic changes in network policies. As software-defined networking controllers are prone to single-point failures, providing security is one of the biggest challenges in this framework. This paper intends to provide an intrusion detection mechanism in both the control plane and data plane to secure the controller and forwarding devices respectively. In the control plane, we imposed a flow-based intrusion detection system that inspects every new incoming flow towards the controller. In the data plane, we assigned a signature-based intrusion detection system to inspect traffic between Open Flow switches using port mirroring to analyse and detect malicious activity. Our flow-based system works with the help of trained, multi-layer machine learning-based classifier, while our signature-based system works with rule-based classifiers using the Snort intrusion detection system. The ensemble feature selection technique we adopted in the flow-based system helps to identify the prominent features and hasten the classification process. Our proposed work ensures a high level of security in the Software-defined networking environment by working simultaneously in both control plane and data plane.

Survey of intrusion detection systems: techniques, datasets and challenges

Cyber-attacks are becoming more sophisticated and thereby presenting increasing challenges in accurately detecting intrusions. Failure to prevent the intrusions could degrade the credibility of security services, e.g. data confidentiality, integrity, and availability. Numerous intrusion detection methods have been proposed in the literature to tackle computer security threats, which can be broadly classified into Signature-based Intrusion Detection Systems (SIDS) and Anomaly-based Intrusion Detection Systems (AIDS). This survey paper presents a taxonomy of contemporary IDS, a comprehensive review of notable recent works, and an overview of the datasets commonly used for evaluation purposes. It also presents evasion techniques used by attackers to avoid detection and discusses future research challenges to counter such techniques so as to make computer systems more secure.