Anomaly-based Intrusion Detection System Research Articles

Detection of Anomalies in the Computer Network Behaviour

Abstract The goal of anomaly-based intrusion detection is to build a system which monitors computer network behaviour and generates alerts if either a known attack or an anomaly is detected. Anomaly-based intrusion detection system detects intrusions based on a reference model which identifies normal behaviour of the computer network and flags an anomaly. Basic challenges in anomaly-based detection are difficulties to identify a ‘normal’ network behaviour and complexity of the dataset needed to train the intrusion detection system. Supervised machine learning can be used to train the binary classifiers in order to recognize the notion of normality. In this paper we present an algorithm for feature selection and instances normalization which reduces the Kyoto 2006+ dataset in order to increase accuracy and decrease time for training, testing and validating intrusion detection systems based on five models: k-Nearest Neighbour (k-NN), weighted k-NN (wk-NN), Support Vector Machine (SVM), Decision Tree, and Feedforward Neural Network (FNN).

A new intrusion detection system based on using non-linear statistical analysis and features selection techniques

The increase in the number of connected devices to the Internet and Internet of Things (IoT) development accompanied a massive increase in the number and types of attacks. Most IoT devices have security vulnerabilities due to their limited computing and storage capabilities and specific protocols. Thus, there is essential to build Intrusion Detection Systems (IDSs) that can detect these threats that affect smart applications. Researchers examined different data mining techniques, statistical analysis techniques, and machine learning (ML) algorithms. In this paper, we propose a novel approach for building an anomaly-based intrusion detection system based on using a non-linear statistical analysis technique called recurrence quantification analysis (RQA). Our approach uses RQA to identify abnormal behavior in an individual feature extracted from a series of packets rather than inspecting the packet's payload. The proposed procedure implies finding the minimum number of features, applying the RQA technique to each effective feature separately, and applying different ML algorithms to classify the RQA measurements resulting from each effective feature. The system's performance was evaluated based on the accuracy and F-score using the UNSW-NB15 dataset. Results show our proposed approach's effectiveness in discovering hidden characteristics in the underlying series of an individual feature that leads to identifying different attacks. Besides, the proposed approach outperforms most previous works using one feature.

An anomaly-based intrusion detection system using recursive feature elimination technique for improved attack detection

The explosive growth of Internet and technologies has resulted in better life-changing services to the society. This drastic change has invariably kept the users connected to the Internet uninterruptedly. The adversaries might exploit this “always-on” Internet connection and would jeopardize the network infrastructure through malicious activities using the networks' hardware and software vulnerabilities. However, the network traffic volume poses difficulties for the cyber analyst to monitor, detect, identify, and quickly respond to the network attacks. Therefore it is paramount to protect the integrity, availability, confidentiality of the network infrastructure, and its digital assets by deploying Intrusion Detection Systems. However, augmenting new techniques avoids overwhelming the analysts from a myriad of events. This research aims to analyze the traffic for the significant features that assist in reliable intrusion detection and minimize the resource requirements and computational complexity of the detection process. The proposed method uses Recursive Feature Elimination technique to rank features based on feature importance and Random Forest algorithm performs the attack classification. The experimental findings show these significant 21 features are useful in anomaly detection with an accuracy of 99.83% for binary attack detection, 99.69% for multiclass classification with an execution time of 20 seconds, and 39 seconds respectively on the NSL-KDD dataset.

Network Anomaly Detection over Streaming Data Using Feature Subset Selection and Ensemble Classifiers

Abstract: Anomaly based Intrusion Detection Systems (IDS) handle ordinary and bizarre way of behaving by look at network traffic in various standard datasets. Goading forIDSs is bountiful of information to process, low discovery rates and steep of deceptions. Considering irregularity design discovery as distinguishing a particular momentwhere the way of behaving of the framework is strange andessentially not the same as past way of behaving. Anticipated on the test results accomplished, we infer that the proposed strategy is a proficient technique for network interruption location. Relationship shows that the proposedmodel is proficient than other existing models as for interruption recognition result.

A Review on Machine Learning Based Approaches of Network Intrusion Detection Systems

The rapid growth of using the Internet raises the possibility of network attacks. In order to secure internal networks, intrusion detection systems are widely employed to address a major research challenge in network security, which aims to efficiently detect unusual access or attacks. To do so, various intrusion detection systems approaches based on the concepts of machine learning algorithms have been developed in the literature to tackle computer security threats. These IDs approaches can be broadly classified into Signature-based Intrusion Detection Systems and Anomaly-based Intrusion Detection Systems. This review paper presents a taxonomy of current intrusion detection systems (IDs), a comprehensive review of significant recent works, and a variety of recent attacks that can be detected in the network environment.

DOC-IDS: A Deep Learning-Based Method for Feature Extraction and Anomaly Detection in Network Traffic.

With the growing diversity of cyberattacks in recent years, anomaly-based intrusion detection systems that can detect unknown attacks have attracted significant attention. Furthermore, a wide range of studies on anomaly detection using machine learning and deep learning methods have been conducted. However, many machine learning and deep learning-based methods require significant effort to design the detection feature values, extract the feature values from network packets, and acquire the labeled data used for model training. To solve the aforementioned problems, this paper proposes a new model called DOC-IDS, which is an intrusion detection system based on Perera’s deep one-class classification. The DOC-IDS, which comprises a pair of one-dimensional convolutional neural networks and an autoencoder, uses three different loss functions for training. Although, in general, only regular traffic from the computer network subject to detection is used for anomaly detection training, the DOC-IDS also uses multi-class labeled traffic from open datasets for feature extraction. Therefore, by streamlining the classification task on multi-class labeled traffic, we can obtain a feature representation with highly enhanced data discrimination abilities. Simultaneously, we perform variance minimization in the feature space, even on regular traffic, to further improve the model’s ability to discriminate between normal and abnormal traffic. The DOC-IDS is a single deep learning model that can automatically perform feature extraction and anomaly detection. This paper also reports experiments for evaluating the anomaly detection performance of the DOC-IDS. The results suggest that the DOC-IDS offers higher anomaly detection performance while reducing the load resulting from the design and extraction of feature values.

Anomaly-based intrusion detection system in IoT using kernel extreme learning machine

Anomaly-based intrusion detection system in IoT using kernel extreme learning machine

Robust anomaly-based intrusion detection system for in-vehicle network by graph neural network framework

Robust anomaly-based intrusion detection system for in-vehicle network by graph neural network framework

Model-based IDS design for ICSs

Industrial systems present security risk factors related to their cyber vulnerabilities. These systems spread out over the world, continue to be targets of attacks. While industrial systems share common vulnerabilities with Information Technology (IT) systems, they tend to have more constraints due to the interaction between cyber and physical systems. Intrusion detection systems give visibility to the system and are considered to be one of the solutions to detect targeting attacks. Hence, it seems relevant to rely on a physical model of the cyber–physical system to obtain an intrusion detection system (IDS) for industrial systems. Most IDSs are based on rules that define how potential attacks are detected. These rules are generally used to describe either the normal system behavior or potential attack scenarios. However, manually creating and maintaining rules for a complex Cyber–Physical system or ICS can prove to be a very tedious and difficult task. This paper proposes a solution to model ICSs and design specific IDSs for ICSs. A model-based IDS rules generator is proposed, which converts a system model into anomaly-based IDS rules. Finally, the effectiveness of the generated rules is evaluated.

IoT Intrusion Detection Using Machine Learning with a Novel High Performing Feature Selection Method

The Internet of Things (IoT) ecosystem has experienced significant growth in data traffic and consequently high dimensionality. Intrusion Detection Systems (IDSs) are essential self-protective tools against various cyber-attacks. However, IoT IDS systems face significant challenges due to functional and physical diversity. These IoT characteristics make exploiting all features and attributes for IDS self-protection difficult and unrealistic. This paper proposes and implements a novel feature selection and extraction approach (i.e., our method) for anomaly-based IDS. The approach begins with using two entropy-based approaches (i.e., information gain (IG) and gain ratio (GR)) to select and extract relevant features in various ratios. Then, mathematical set theory (union and intersection) is used to extract the best features. The model framework is trained and tested on the IoT intrusion dataset 2020 (IoTID20) and NSL-KDD dataset using four machine learning algorithms: Bagging, Multilayer Perception, J48, and IBk. Our approach has resulted in 11 and 28 relevant features (out of 86) using the intersection and union, respectively, on IoTID20 and resulted 15 and 25 relevant features (out of 41) using the intersection and union, respectively, on NSL-KDD. We have further compared our approach with other state-of-the-art studies. The comparison reveals that our model is superior and competent, scoring a very high 99.98% classification accuracy.

Intelligent and Improved Self-Adaptive Anomaly based Intrusion Detection System for Networks

With the advent of digital technology, computer networks have developed rapidly at an unprecedented pace contributing tremendously to social and economic development. They have become the backbone for all critical sectors and all the top Multi-National companies. Unfortunately, security threats for computer networks have increased dramatically over the last decade being much brazen and bolder. Intrusions or attacks on computers and networks are activities or attempts to jeopardize main system security objectives, which called as confidentiality, integrity and availability. They lead mostly in great financial losses, massive sensitive data leaks, thereby decreasing efficiency and the quality of productivity of an organization. There is a great need for an effective Network Intrusion Detection System (NIDS), which are security tools designed to interpret the intrusion attempts in incoming network traffic, thereby achieving a solid line of protection against inside and outside intruders. In this work, we propose to optimize a very popular soft computing tool prevalently used for intrusion detection namely Back Propagation Neural Network (BPNN) using a novel machine learning framework called “ISAGASAA”, based on Improved Self-Adaptive Genetic Algorithm (ISAGA) and Simulated Annealing Algorithm (SAA). ISAGA is our variant of standard Genetic Algorithm (GA), which is developed based on GA improved through an Adaptive Mutation Algorithm (AMA) and optimization strategies. The optimization strategies carried out are Parallel Processing (PP) and Fitness Value Hashing (FVH) that reduce execution time, convergence time and save processing power. While, SAA was incorporated to ISAGA in order to optimize its heuristic search. Experimental results based on Kyoto University benchmark dataset version 2015 demonstrate that our optimized NIDS based BPNN called “ANID BPNN-ISAGASAA” outperforms several state-of-art approaches in terms of detection rate and false positive rate. Moreover, improvement of GA through FVH and PP saves processing power and execution time. Thus, our model is very much convenient for network anomaly detection.

Dealing with Security Alert Flooding: Using Machine Learning for Domain-independent Alert Aggregation

Intrusion Detection Systems (IDS) secure all kinds of IT infrastructures through automatic detection of malicious activities. Unfortunately, they are known to produce large numbers of alerts that often become overwhelming for manual analysis. Therefore, aggregation methods have been developed for filtering, grouping, and correlating alerts. However, existing techniques either rely on manually defined attack scenarios or require specific alert formats, such as IDMEF that include IP addresses. This makes the application of existing aggregation methods infeasible for alerts from host-based or anomaly-based IDSs that frequently lack such network-related data. In this paper, we therefore present a domain-independent alert aggregation technique. We introduce similarity measures and merging strategies for arbitrary semi-structured alerts and alert groups. Based on these metrics and techniques we propose an incremental procedure for the generation of abstract alert patterns that enable continuous classification of incoming alerts. Evaluations show that our approach is capable of reducing the number of alert groups for human review by around \( 80\% \) and assigning attack classifiers to the groups with true positive rates of \( 80\% \) and false positive rates lower than \( 5\% \) .

Application of Representation Learning-Based Chronological Modeling for Network Intrusion Detection

An autoencoder has the potential to overcome the limitations of current intrusion detection methods by recognizing benign user activity rather than differentiating between benign and malicious activity. However, the line separating them is quite blurry with a significant overlap. The first part of this study aims to investigate the rationale behind this overlap. The results suggest that although a subset of traffic cannot be separated without labels, timestamps have the potential to be leveraged for identification of activity that does not conform to the normal or expected behavior of the network. The second part aims to eliminate dependence on visual-inspections by exploring automation. The trend of errors for HTTP traffic was modeled chronologically using resampled data and moving averages. This model successfully identified attacks that had orchestrated over HTTP within their respective time slots. These results support the hypothesis that it is technically feasible to build an anomaly-based intrusion detection system where each individual observation need not be categorized.

Investigation on Intrusion Detection Systems (IDSs) in IoT

In smart environments, comfort and efficiency are important goals in terms of the quality of human life. Recent developments in Internet of Things (IoT) technology have made it possible to design smart environments. IoT-based smart environments are concerned with security and privacy as key issues. Systems based on the IoT pose a security threat to smart environments. In order to prevent IoT-related security attacks which take advantage of some of these security vulnerabilities, intrusion detection systems (IDSs) designed for IoT environments are crucial. Data generated by connected objects in the age of the IoT provides the basis for big data analytics, which could be employed to identify patterns and identifies anomalies in data. In order to detect intrusions, most cyber security systems employ IDSs, which are used by a variety of techniques and architectures. As opposed to signature-based IDS, anomaly-based IDS learns the normal pattern of system behavior and alerts on abnormal events that occur, as opposed to monitoring monitored events against a database of known intrusion experiences. This paper focuses on the IDS implementation on the IoT network. The use of sensor devices to collect data from smart grid environments has led to smart grids becoming the preferred intrusion target due to the IoTs using advanced information technology. Clouds are data storage systems that provide a variety of smart infrastructure services, such as smart homes and smart buildings, over the internet. A deep learning-based intrusion detection system for the Internet of Things requires consideration of key design principles presented in this paper.

A novel hybrid-based approach of snort automatic rule generator and security event correlation (SARG-SEC).

The rapid advanced technological development alongside the Internet with its cutting-edge applications has positively impacted human society in many aspects. Nevertheless, it equally comes with the escalating privacy and critical cybersecurity concerns that can lead to catastrophic consequences, such as overwhelming the current network security frameworks. Consequently, both the industry and academia have been tirelessly harnessing various approaches to design, implement and deploy intrusion detection systems (IDSs) with event correlation frameworks to help mitigate some of these contemporary challenges. There are two common types of IDS: signature and anomaly-based IDS. Signature-based IDS, specifically, Snort works on the concepts of rules. However, the conventional way of creating Snort rules can be very costly and error-prone. Also, the massively generated alerts from heterogeneous anomaly-based IDSs is a significant research challenge yet to be addressed. Therefore, this paper proposed a novel Snort Automatic Rule Generator (SARG) that exploits the network packet contents to automatically generate efficient and reliable Snort rules with less human intervention. Furthermore, we evaluated the effectiveness and reliability of the generated Snort rules, which produced promising results. In addition, this paper proposed a novel Security Event Correlator (SEC) that effectively accepts raw events (alerts) without prior knowledge and produces a much more manageable set of alerts for easy analysis and interpretation. As a result, alleviating the massive false alarm rate (FAR) challenges of existing IDSs. Lastly, we have performed a series of experiments to test the proposed systems. It is evident from the experimental results that SARG-SEC has demonstrated impressive performance and could significantly mitigate the existing challenges of dealing with the vast generated alerts and the labor-intensive creation of Snort rules.

A Vote-Based Architecture to Generate Classified Datasets and Improve Performance of Intrusion Detection Systems Based on Supervised Learning

An intrusion detection system (IDS) is an important tool to prevent potential threats to systems and data. Anomaly-based IDSs may deploy machine learning algorithms to classify events either as normal or anomalous and trigger the adequate response. When using supervised learning, these algorithms require classified, rich, and recent datasets. Thus, to foster the performance of these machine learning models, datasets can be generated from different sources in a collaborative approach, and trained with multiple algorithms. This paper proposes a vote-based architecture to generate classified datasets and improve the performance of supervised learning-based IDSs. On a regular basis, multiple IDSs in different locations send their logs to a central system that combines and classifies them using different machine learning models and a majority vote system. Then, it generates a new and classified dataset, which is trained to obtain the best updated model to be integrated into the IDS of the companies involved. The proposed architecture trains multiple times with several algorithms. To shorten the overall runtimes, the proposed architecture was deployed in Fed4FIRE+ with Ray to distribute the tasks by the available resources. A set of machine learning algorithms and the proposed architecture were assessed. When compared with a baseline scenario, the proposed architecture enabled to increase the accuracy by 11.5% and the precision by 11.2%.

Anomaly-based intrusion detection system for IoT networks through deep learning model

The Internet of Things (IoT) idea has been developed to enhance people's lives by delivering a diverse range of smart interconnected devices and applications in several domains. However, security threats are main critical challenges for the devices in an IoT environment. Many approaches have been proposed to secure IoT appliances in state of the art, still advancement is desirable. Machine learning has demonstrated a capability to detect patterns when other methodologies have collapsed. One advanced method to enhance IoT security is to employ deep learning. This formulates a seamless option for anomaly-based detection. This paper presents a CNN-based approach for anomaly-based intrusion detection systems (IDS) that takes advantage of IoT's power, providing qualities to efficiently examine whole traffic across the IoT. The proposed model shows ability to detect any possible intrusion and abnormal traffic behavior. The model is trained and tested using the NID Dataset and BoT-IoT datasets and achieved an accuracy of 99.51% and 92.85%, respectively.

A new Wrapper Feature Selection model for Anomaly based Intrusion Detection Systems

A new Wrapper Feature Selection model for Anomaly based Intrusion Detection Systems

MTH-IDS: A Multitiered Hybrid Intrusion Detection System for Internet of Vehicles

Modern vehicles, including connected vehicles and autonomous vehicles, nowadays involve many electronic control units connected through intra-vehicle networks to implement various functionalities and perform actions. Modern vehicles are also connected to external networks through vehicle-to-everything technologies, enabling their communications with other vehicles, infrastructures, and smart devices. However, the improving functionality and connectivity of modern vehicles also increase their vulnerabilities to cyber-attacks targeting both intra-vehicle and external networks due to the large attack surfaces. To secure vehicular networks, many researchers have focused on developing intrusion detection systems (IDSs) that capitalize on machine learning methods to detect malicious cyber-attacks. In this paper, the vulnerabilities of intra-vehicle and external networks are discussed, and a multi-tiered hybrid IDS that incorporates a signature-based IDS and an anomaly-based IDS is proposed to detect both known and unknown attacks on vehicular networks. Experimental results illustrate that the proposed system can detect various types of known attacks with 99.99% accuracy on the CAN-intrusion-dataset representing the intra-vehicle network data and 99.88% accuracy on the CICIDS2017 dataset illustrating the external vehicular network data. For the zero-day attack detection, the proposed system achieves high F1-scores of 0.963 and 0.800 on the above two datasets, respectively. The average processing time of each data packet on a vehicle-level machine is less than 0.6 ms, which shows the feasibility of implementing the proposed system in real-time vehicle systems. This emphasizes the effectiveness and efficiency of the proposed IDS.

Anomaly Detection Framework in Fog-to-Things Communication for Industrial Internet of Things

The rapid development of the Internet of Things (IoT) in the industrial domain has led to the new term the Industrial Internet of Things (IIoT). The IIoT includes several devices, applications, and services that connect the physical and virtual space in order to provide smart, cost-effective, and scalable systems. Although the IIoT has been deployed and integrated into a wide range of industrial control systems, preserving security and privacy of such a technology remains a big challenge. An anomaly-based Intrusion Detection System (IDS) can be an effective security solution for maintaining the confidentiality, integrity, and availability of data transmitted in IIoT environments. In this paper, we propose an intelligent anomaly-based IDS framework in the context of fog-to-things communications to decentralize the cloud-based security solution into a distributed architecture (fog nodes) near the edge of the data source. The anomaly detection system utilizes minimum redundancy maximum relevance and principal component analysis as the featured engineering methods to select the most important features, reduce the data dimensionality, and improve detection performance. In the classification stage, anomaly-based ensemble learning techniques such as bagging, LPBoost, RUSBoost, and Adaboost models are implemented to determine whether a given flow of traffic is normal or malicious. To validate the effectiveness and robustness of our proposed model, we evaluate our anomaly detection approach on a new driven IIoT dataset called X-IIoTID, which includes new IIoT protocols, various cyberattack scenarios, and different attack protocols. The experimental results demonstrated that our proposed anomaly detection method achieved a higher accuracy rate of 99.91% and a reduced false alarm rate of 0.1% compared to other recently proposed techniques.