Adaptive multi-view transformer ensemble for intrusion detection: Addressing data imbalance and enhancing attack classification

Traffic classification is an automated technique that divides computer network traffic into several categories depending on different factors like protocol or port number. In a complicated context, traffic categorization is an important tool for network and system security. A monitoring system called intrusion detection looks for abnormal activity and sends out notifications. In order to safeguard a system from network-based attacks, Network Intrusion Detection Systems (NIDS) play a crucial role in monitoring and analyzing network traffic. Active and passive intrusion detection systems (IDS), network intrusion detection systems (NIDS), host intrusion detection systems (HIDS), knowledge-based (signature-based) IDS, and behaviorbased (anomaly-based) IDS are some of the numerous types of intrusion detection systems (IDS). Passive IDS is just designed to monitor and analyze network traffic behaviour and notify an operator of potential vulnerabilities and attacks, whereas Active IDS is also known as Intrusion Detection and Prevention System. A network's malicious traffic is identified using a network-based intrusion detection system (NIDS). A host-based IDS monitors system activity and seeks for indications of abnormal behaviour. For networks with unidentified traffic, the intrusion detection system designed using flow and payload statistical characteristics and clustering approach needs additional clusters. The present intrusion detection system however is affected by false alarm rate, poor detection rate, imbalanced datasets and response time which lead to misclassification of intrusions in various scenarios. Hence, there is a requirement for developing an automated intrusion detection system that works well in different scenarios. The proposed system uses supervised and unsupervised intrusion detection and classification methods to increase the classification accuracy. To categorize the intrusions, dimensionality reduction strategies are used in conjunction with the classification procedure of logistic regression. Performance of intrusion detection system using PCA as dimensionality reduction algorithm has been evaluated with different classifiers such as Logistic Regression (LR), K-Nearest Neighbors (K-NN), Random Forest (RF), Support Vector Machine (Kernel SVM), Decision Tree (DT) using CIC IDS 2022 dataset. An automated way to detect intrusions has been proposed with cluster formation using adaptive weight butterfly optimization algorithm.

Chapter 2 - Cisco Intrusion Detection

RNNIDS: Enhancing network intrusion detection systems through deep learning

This paper describes the ESIDE-Depian intrusion detection and prevention system, which uses Bayesian structural and parametric learning and also evidence propagation and adaptation, in order to improve the accuracy and manageability of network intrusion detection systems (NIDS). Current NIDS do not consider the two main detection paradigms, i.e. misuse detection and anomaly detection, in an unified style, so the analysis is not inherently complete. Besides, historical data are not generally used, neither for analysis nor for sequential adaptation of the knowledge representation models used for detection; hence this wealthy information about the essence and the potential trends of the target system is not commonly considered. Thus, by the generalized use of Bayesian belief networks, ESIDE-Depian achieves the main goal of detecting and preventing both well-known and also zero-day attacks with excellent results, by means of unified real-time analysis of network traffic.

This paper describes the ESIDE-Depian intrusion detection and prevention system, which uses Bayesian structural and parametric learning and also evidence propagation and adaptation, in order to improve the accuracy and manageability of network intrusion detection systems (NIDS). Current NIDS do not consider the two main detection paradigms, i.e. misuse detection and anomaly detection, in an unified style, so the analysis is not inherently complete. Besides, historical data are not generally used, neither for analysis nor for sequential adaptation of the knowledge representation models used for detection; hence this wealthy information about the essence and the potential trends of the target system is not commonly considered. Thus, by the generalized use of Bayesian belief networks, ESIDE-Depian achieves the main goal of detecting and preventing both well-known and also zero-day attacks with excellent results, by means of unified real-time analysis of network traffic.

Accurate identification of network intrusions is one of the biggest challenges of Network Intrusion Detection (NID) systems. In recent years Machine learning classification techniques have been used to precisely identify network intrusion. However, the multi class distribution in network intrusion detection system has found to be highly skewed, leading to classification accuracy problem due to class imbalance data set. The work presented in this paper not only explores the role of the attribute selection in improving classification accuracy but also investigates the problem of class imbalance using the Synthetic Minority Over-sampling (SMOTE) and under sampling of major classes. The classification performance is then evaluated over several types of classifiers. The outcome of this work is that for the class imbalance data set the under-sampling technique is more effective than SMOTE in detecting minor classes. It has also found during this research work that the decision tree algorithms (JRIP) and Naïve Bayes are more accurate classifiers as compared to the Radial basis neural network and support vector machine. However no single algorithm can be used for the classification of multiclass and it is proposed in this research work that combination of classifier consisting of Naïve Bayes and JRIP could be used for the classification of minor classes in an imbalance class data set of intrusion detection system.

Network security is becoming an increasingly important issue, since the rapid development of the Internet. Network Intrusion Detection System (IDS), as the main security defending technique, is widely used against such malicious attacks. Data mining and machine learning technology has been extensively applied in network intrusion detection and prevention systems by discovering user behavior patterns from the network traffic data. Association rules and sequence rules are the main technique of data mining for intrusion detection. Considering the classical Apriori algorithm with bottleneck of frequent itemsets mining, we propose a Length-Decreasing Support to detect intrusion based on data mining, which is an improved Apriori algorithm. Experiment results indicate that the proposed method is efficient.

In the ubiquitously connected world of IT infrastructure, Intrusion Detection System (IDS) plays vital role. IDS is considered as a critical component of security infrastructure and is implemented either through hardware or software devices and can detect malicious activities in a networked environment. To detect or prevent network attacks, Network Intrusion Detection (NID) system may be equipped with machine learning algorithms to achieve better accuracy and faster detection speed. Analyzing different attacks effectively through Dimensionality Reduction Algorithms is an efficient mechanism. The significance of these algorithms is they improvise feature selection from huge datasets. Also through this the learning speed is enhanced. Speed is a crucial parameter in the success of network intrusion detection systems for defending reactions. In this paper open source datasets Knowledge Discovery in Databases (KDD CUP) dataset and 10% KDD CUP dataset are employed for experimentation. These datasets are provided to Dimensionality Reduction Algorithms like Principal Component Analysis (PCA), Linear Discriminate Analysis (LDA) and Kernel PCA with different kernels and classified with Logistic Regression classification algorithm for procuring accurate results. Further to boost up the accuracy achieved so far K-fold algorithm is utilized. Finally a comparative study of different accuracy results is done by using K-fold algorithm and also without the usage of this algorithm. The empirical study on KDD CUP data confirms the effectiveness of the proposed scheme. In this paper we discovered the combination of multiple dimensionality reduction algorithm such as PCA , LDA and Kernel PCA with classification algorithm and this combination of algorithm gives best result. Our study will help out the researchers to uncover critical area such as intrusion detection in network traffic environment. The results what we identified will be very much helpful for researchers for their future research on KDD CUP dataset. In this the new theory will be arrived by this research that the best accuracy achieved by PCA with 10% KDD CUP dataset experimental results without KFold attained 98% and with KFold attained 99%. LDA with 10% KDD CUP Dataset experimental results without KFold attained 98% and with KFold attained 99%.

One of the major problems faced by anomaly based Network Intrusion Detection (NID) systems is the high number of false positives. False positives refer to the false detection of normal behavior as malicious behavior. Artificial Immune Systems (AISs) also fall under the category of anomaly based-NID systems. AIS presented in this paper is as a victim-end filter, consisting of detectors distributed on the network, which distinguishes normal traffic from malicious traffic. In this work, we focus on TCP-SYN flood based Distributed Denial of Services (DDoS) attacks. Light Weight Intrusion Detection System (LISYS) provides the basic framework for AIS based NID systems. AISs normally utilize the negative selection algorithm in thymus action to tolerize the detectors to normal traffic so they may not detect normal traffic as malicious traffic. We propose and implement `extended thymus action' model to improve this characteristic of AIS. Results verify that our model significantly reduces false positives which is a major concern in anomaly-based NID systems.

Machine learning (ML)-based Network Intrusion Detection Systems (NIDSs) can classify each network’s flow behavior as benign or malicious by detecting heterogeneous features, including both categorical and numerical features. However, the present ML-based NIDSs are deemed insufficient in terms of their ability to generalize, particularly in changing network environments such as the Internet of Things (IoT)-based smart home. Although IoT devices add so much to home comforts, they also introduce potential risks and vulnerabilities. Recently, many NIDS studies on other IoT scenarios, such as the Internet of Vehicles (IoV) and smart cities, focus on utilizing the telemetry data of IoT devices for IoT intrusion detection. Because when IoT devices are under attack, their abnormal telemetry data values can reflect the anomaly state of those devices. Those telemetry data-based IoT NIDS methods detect intrusion events from a different view, focusing on the attack impact, from the traditional network traffic-based NIDS, which focuses on analyzing attack behavior. The telemetry data-based NIDS is more suitable for IoT devices without built-in security mechanisms. Considering the smart home IoT scenario, which has a smaller scope and a limited number of IoT devices compared to other IoT scenarios, both NIDS views can work independently. This motivated us to propose a novel ML-based NIDS to combine the network traffic-based and telemetry data-based NIDS together. In this paper, we propose a Transformer-based IoT NIDS method to learn the behaviors and effects of attacks from different types of data that are generated in the heterogeneous IoT environment. The proposed method utilizes a self-attention mechanism to learn contextual embeddings for input network features. Based on the contextual embeddings, our method can solve the feature set challenge, including both continuous and categorical features. Our method is the first to utilize both network traffic data and IoT sensors’ telemetry data at the same time for intrusion detection. Experiments reveal the effectiveness of our method on a realistic network traffic intrusion detection dataset named ToN_IoT, with an accuracy of 97.95% for binary classification and 95.78% for multiple classifications on pure network data. With the extra IoT information, the performance of our method has been improved to 98.39% and 97.06%, respectively. A comparative study with existing works shows that our method can achieve state-of-the-art performance on the ToN_IoT dataset.

Today's anomaly-based network intrusion detection systems (IDSs) are plagued with detecting new and unknown attacks. The review of the literature builds ideas for researching the problem of detecting these attacks using multi-layered feed forward neural network (MLFFNN) IDSs. The scope of the paper focused on a review of the literature from primarily 2008 to the present found in peer-review and scholarly journals. A key word search was used to compare and contrast the literature to find strengths, weaknesses and gaps. The significance of the research found that further work is needed to improve the performance and convergence rates of MLFFNN IDSs. This literature review contributes to the area of intrusion detection by looking at the effects of architecture, algorithms, and input data on the performance and convergence rates of MLFFNN IDSs.

The traditional methods and strategies for educational- learning training in universities do not respond to the learning needs of students applying to "special training courses" (Corsi Abilitanti speciali) that are already experienced teachers and workers, and that for various reasons and motivations are not able to attend classroom courses. In this environment, SSIS Umbria (post-graduate training for high school teaching qualification) is experimenting a form of blended e-learning aimed at supporting classroom lessons, during which a network approach to knowledge and some forms of collaboration and cooperation in know-how building are integrated. Experimentation has proposed integration of on-line activities that students are able to carry out on their own, and other activities to be performed in small learning groups, consistent with the class courses envisaged for teacher training. The experimentation has also allowed to assess, through log analysis, the degree of interest of participants and distribution of their time between the various activities envisaged by the e-learning software. The method for evaluation of participant activities has not been based on a simple recording of the on-line connection time, but rather on the assignation of a previously-set time "token" in minutes to each task performed, according to the commitment forecast for the accomplishment of each single activity. This type of monitoring has been organized by backing up software time-counting of "minute-amounts" coming from on-line practice with tutor control, in order to supervise the respect of an accurate netiquette.

Zero-Day attack detection in Network Intrusion Detection Systems (NIDS) refers to the ability to identify previously unseen attack patterns during testing without having been explicitly trained on those specific attacks, utilizing learned features from other known attacks. In this paper, we propose a Deep Reinforcement Learning (DRL)-based NIDS designed for Zero-Day attack detection. We use a stacked LSTM architecture to extend the learning capabilities of the DRL agent. We apply several oversampling techniques to handle the issue of class imbalance since the zero-day attack datasets are not as abundant. We use some of the most widely available benchmark datasets in NIDS domain, which all together cover a wide range of attack types, such as reconnaissance, ddoS, infiltration, injection, password attacks, brute force, dos, backdoor, and benign traffic. For example, we converted attacks to 1 and benign traffic to 0, then excluded certain attack categories (DoS and Backdoor) from the training dataset while keeping them in the test dataset. This makes those attack types zero-day attacks, as they are entirely unseen during training. We also compare which data balancing technique works better among K-means SMOTE, SMOTE, Borderline-SMOTE and ADASYN on the performance of our DRL agent. We then demonstrate how powerful our agent is by validating many datasets for remarkable success in detecting both known and unknown attacks in a zero-day manner. Our work has been made publicly available on GitHub (<uri>https://github.com/codewithkhurshed/ZDAD</uri>) to support researchers in advancing zero-day attack detection in NIDS.

Network Intrusion Detection System (NIDS) is considered as one of the last defense mechanisms for any organization. NIDS can be broadly classified into two approaches: misuse-based detection and anomaly-based detection. Misuse-based intrusion detection builds a database of the well-defined patterns of the attacks that exploit weaknesses in systems and network protocols, and uses that database to identify the intrusions. Although this approach can detect all the attacks included in the database, it leads to false negative errors where any new attack not included in that database can’t be detected. The other approach is the anomaly-based NIDS which is developed to emulate the Human Immune System (HIS) and overcome the limitation of the misuse-based approach. The anomaly-based detection approach is based on Negative Selection (NS) mechanism. NS is based on building a database of the normal self patterns, and identifying any pattern not included in that database as a non-self pattern and hence the intrusion is detected. Unfortunately, NS concept has also its drawbacks. Although any attack pattern can be detected as a non-self pattern and this leads to low false negative rate, non-self patterns would not necessarily indicate the existence of intrusions. So, NS has a high false positive error rate caused from that assumption. Danger Theory (DT) is a new concept in HIS, which shows that the response mechanism in HIS is more complicated and beyond the simple NS concept. So, is it possible to utilize the DT to minimize the high false positive detection rate of NIDS? This paper answers this question by developing a prototype for NIDS based on DT and evaluating that prototype using DARPA99 Intrusion Detection dataset.

iNIDS: SWOT Analysis and TOWS Inferences of State-of-the-Art NIDS solutions for the development of Intelligent Network Intrusion Detection System