Real-time Network Intrusion Detection Research Articles

A Comprehensive Review of Dimensionality Reduction Techniques for Real time Network Intrusion Detection with Applications in Cybersecurity

This paper reviews popular signature and anomaly-based intrusion detection systems (IDS). Dimensionality reduction techniques (DRT) are used to increase the efficiency of such systems for real-time operation. Autoencoder-based IDS is rapidly gaining in popularity, primarily due to its inherent ability to denoise data and reduce dimensionality. In addition to the efficiency, we also look at the classification techniques used by various authors, and the overall impact of a model in terms of performance metrics. This article is written for novices in cybersecurityto get a jumpstart on the latest IDS algorithms. The purpose is to give useful insights into the broad and progressive view of various techniques in wide use, expose high-impact future research areas and to summarize classic IDS methods and feature selection techniques.

Towards Real-time Network Intrusion Detection with Image-based Sequential Packets Representation

Towards Real-time Network Intrusion Detection with Image-based Sequential Packets Representation

LDA-ID: An LDA-based framework for real-time network intrusion detection

Network intrusion poses a severe threat to the Internet. However, existing intrusion detection models cannot effectively distinguish different intrusions with high-degree feature overlap. In addition, efficient real-time detection is an urgent problem. To address the two above problems, we propose a Latent Dirichlet Allocation topic model-based framework for real-time network Intrusion Detection (LDA-ID), consisting of static and online LDA-ID. The problem of feature overlap is transformed into static LDA-ID topic number optimization and topic selection. Thus, the detection is based on the latent topic features. To achieve efficient real-time detection, we design an online computing mode for static LDA-ID, in which a parameter iteration method based on momentum is proposed to balance the contribution of prior knowledge and new information. Furthermore, we design two matching mechanisms to accommodate the static and online LDA-ID, respectively. Experimental results on the public NSL-KDD and UNSW-NB15 datasets show that our framework gets higher accuracy than the others.

A Network Intrusion Detection Model Based on Principal Component Analysis and Random Forest

Network intrusion data has the characteristics of high dimension, nonlinearity and redundancy. To solve the problem of low detection rate of traditional dimensionality reduction and detection methods, a network intrusion detection method based on principal component analysis combined with random forest is proposed. Firstly, the data dimension of network intrusion is reduced by principal component analysis to eliminate redundant information between data, and then the processed data is classified and trained by using random forest classifier. The algorithm is verified by the network intrusion NSL_KDD dataset. The experimental results show that compared with other network intrusion detection methods, the method has fast learning speed, high detection accuracy, low false negative rate and low false positive rate. An efficient, real-time and good network intrusion detection method.

Real-time network intrusion detection using deferred decision and hybrid classifier

The network intrusion detection system needs to detect intrusions in real-time without delay when an intrusion is attempted. To this end, various approaches, such as packet-based classifiers and session-based classifiers, have been developed, but there is a limitation in effectively reducing the detection delay. This study proposes a packet-based machine-learning model with a deferred decision and a single packet integration classifier with cumulative features based on this model to solve this problem fundamentally. A hybrid classifier that uses packet-based features and session-based features was combined to develop a new machine-learning-based network intrusion detection system. Unlike conventional methods, the proposed method could achieve very high detection accuracy and detect intrusions in real-time by minimizing the detection delay. In addition, it reached very high scalability, so it is easy to support high traffic rates and high concurrent session capacity. The performance was compared with various existing methods on various datasets. Through this, the superiority of the proposed method was proven. Overall, the current network can be protected more securely and quickly by applying the proposed method.

A real-time adaptive network intrusion detection for streaming data: a hybrid approach

A real-time adaptive network intrusion detection for streaming data: a hybrid approach

A Continuous Learning Approach for Real-Time Network Intrusion Detection.

Network intrusion detection is becoming a challenging task with cyberattacks that are becoming more and more sophisticated. Failing the prevention or detection of such intrusions might have serious consequences. Machine learning approaches try to recognize network connection patterns to classify unseen and known intrusions but also require periodic re-training to keep the performances at a high level. In this paper, a novel continuous learning intrusion detection system, called Soft-Forgetting Self-Organizing Incremental Neural Network (SF-SOINN), is introduced. SF-SOINN, besides providing continuous learning capabilities, is able to perform fast classification, is robust to noise, and it obtains good performances with respect to the existing approaches. The main characteristic of SF-SOINN is the ability to remove nodes from the neural network based on their utility estimate. SF-SOINN has been validated on the well-known NSL-KDD and CIC-IDS-2017 intrusion detection datasets as well as on some artificial data to show the classification capability on more general tasks.

A Study on the Efficacy of Deep Reinforcement Learning for Intrusion Detection

The world has witnessed a fast-paced digital transformation in the past decade, giving rise to all-connected environments. While the increasingly widespread availability of networks has benefited many aspects of our lives, providing the necessary infrastructure for smart autonomous systems, it has also created a large cyber attack surface. This has made real-time network intrusion detection a significant component of any computerized system. With the advances in computer hardware architectures with fast, high-volume data processing capabilities and the developments in the field of artificial intelligence, deep learning has emerged as a significant aid for achieving accurate intrusion detection, especially for zero-day attacks. In this paper, we propose a deep reinforcement learning-based approach for network intrusion detection and demonstrate its efficacy using two publicly available intrusion detection datasets, namely NSL-KDD and UNSW-NB15. The experiment results suggest that deep reinforcement learning has significant potential to provide effective intrusion detection in the increasingly complex networks of the future.

Towards Effective Network Intrusion Detection: From Concept to Creation on Azure Cloud

Network Intrusion Detection is one of the most researched topics in the field of computer security. Hacktivists use sophisticated tools to launch numerous attacks that hamper the confidentiality, integrity and availability of computer resources. There is an incessant need to safeguard these resources to avoid further damage. In the proposed study, we have presented a meta-classification approach using decision jungle to perform both binary and multiclass classification. We have established the robustness of our approach by configuring an optimal set of hyper-parameters coupled with relevant feature subsets using a production-ready environment namely Azure machine learning. We have validated the efficiency of the proposed design using three contemporary datasets namely UNSW NB-15, CICIDS 2017, and CICDDOS 2019. We could achieve an accuracy of 99.8% pertaining to UNSW NB-15 whereas the accuracy in the case of CICIDS 2017 and CICDDOS 2019 datasets has been 98% and 97% respectively. A distinctive ability of the proposed model lies in its finesse to detect thirty-three modern attack types considerably well. Unlike conventional stacking ensembles, the proposed solution relies on a train-test ratio of 40:60 to establish the legitimacy of predictions. We also conducted statistical significance tests to compare the performance of classifiers involved in the study. To extend the functionalities further, we have automated the proposed model that can be a reliable candidate for real-time network intrusion detection.

Designing online network intrusion detection using deep auto-encoder Q-learning

Because of the increasing application of reinforcement learning (RL), particularly deep Q-learning algorithm, research organizations utilize it with increasing frequency. The prediction of cyber vulnerability and development of efficient real-time online network intrusion detection (NID) systems are progressions toward becoming RL-powered. An open issues in NID is the model design and prediction of real-time online data composed of a series of time-related feature patterns. There have been concerns regarding the operation of the developed systems because cyber-attack scenarios vary continuously to circumvent NID. These issues have been related to the human interaction significance and the decrease in accuracy verification. Therefore, we employ an RL that permits a deep auto-encoder in the Q-network (DAEQ-N). The proposed DAEQ-N attempts to achieve the maximum prediction accuracy in online learning systems into which continuous behavior patterns are fed and which are trained with more significant weights by classifying it as either “normal” or “anomalous.”

Design and Analysis of Real-time Network Intrusion Detection and Prevention System using Open Source Tools

detection and prevention is one of the most important and fundamental task in an organization's computer network. Commercially available intrusion detection and prevention systems are costly and overkill for small and medium sized organizations. This paper describes the design and analysis of a network intrusion detection system (NIDS) and network intrusion prevention system (NIPS) using open source tools. The study also describes an open source Database to store the alerts and an open source front end management console application to view the alerts and logs from the proposed Database in any of the modern day web browser. In this particular research Snort was used as an NIDS to detect intrusions and attacks. Snort is a popular open source NIDS with signature based rules for detecting thousands of known attacks. The rules are regularly updated by Snort team to include new attacks and intrusions. SnortSam was used as an NIPS to act upon the alerts detected by Snort. SnortSam blocks the intrusions by sending intruders and attacker's source IP addresses to firewall in real time. MySQL was used as the Database to store alerts and BASE (Basic Analysis and Security Engine) was chosen as the open source management console application. Juniper Networks switch EX-3200 and Firewall SSG-20 were used as the network devices for connectivity and working of the system. Any other vendor network devices can also effectively be used in design and configuration of the system. The design successfully detected and prevented network intrusions and same can be implemented in any small and medium sized organization for protection of their Computer Networks.

Selection of Candidate Support Vectors in incremental SVM for network intrusion detection

In an Incremental Support Vector Machine classification, the data objects labelled as non-support vectors by the previous classification are re-used as training data in the next classification along with new data samples verified by Karush–Kuhn–Tucker (KKT) condition. This paper proposes Half-partition strategy of selecting and retaining non-support vectors of the current increment of classification – named as Candidate Support Vectors (CSV) – which are likely to become support vectors in the next increment of classification. This research work also designs an algorithm named the Candidate Support Vector based Incremental SVM (CSV-ISVM) algorithm that implements the proposed strategy and materializes the whole process of incremental SVM classification. This work also suggests modifications to the previously proposed concentric-ring method and reserved set strategy. Performance of the proposed method is evaluated with experiments and also by comparing it with other ISVM techniques. Experimental results and performance analyses show that the proposed algorithm CSV-ISVM is better than general ISVM classifications for real-time network intrusion detection.

A New Data Stream Clustering Approach about Intrusion Detection

Intrusion detection is one of the most important techniques for protecting network security. In addition, intrusion detection model can be used to recognize real-time pattern, which has important practical significance for real-time intrusion detection. However, due to the sheer speed and scale of the data, data points must often be analyzed in real time. The one-pass-through requirement and the lack of efficient clustering algorithms to identify intrusion patterns limit the power and scalability of this approach. A data stream clustering algorithm is proposed for real-time network intrusion detection. By introducing the new hashing mechanism, the method can quickly find the clustering patterns in the data stream. The method significantly reduces the false alarm rate of intrusion detection, and improves the performance of intrusion detection system.

A real-time network intrusion detection system for large-scale attacks based on an incremental mining approach

None of the previously proposed Network Intrusion Detection Systems (NIDSs), which are subject to fuzzy association rules, can meet real-time requirements because they all apply static mining approaches. This study proposed a real-time NIDS with incremental mining for fuzzy association rules. By consistently comparing the two rule sets, one mined from online packets and the other mined from training attack-free packets, the proposed system can render a decision every 2 seconds. Thus, compared with traditional static mining approaches, the proposed system can greatly improve efficiency from offline detection to real-time online detection. Since the proposed system derives features from packet headers only, like the previous works based on fuzzy association rules, large-scale attack types are focused. Many DoS attacks were experimented in this study. Experiments were performed to demonstrate the excellent effectiveness and efficiency of the proposed system. The system may not cause false alarms because normal programs supposedly would not generate enough mal-formatted packets, or packets that violate normal network protocols.

Enriching network security analysis with time travel

In many situations it can be enormously helpful to archive the raw contents of a network traffic stream to disk, to enable later inspection of activity that becomes interesting only in retrospect. We present a Time Machine (TM) for network traffic that provides such a capability. The TM leverages the heavy-tailed nature of network flows to capture nearly all of the likely-interesting traffic while storing only a small fraction of the total volume. An initial proof-of-principle prototype established the forensic value of such an approach, contributing to the investigation of numerous attacks at a site with thousands of users. Based on these experiences, a rearchitected implementation of the system provides flexible, highperformance traffic stream capture, indexing and retrieval, including an interface between the TM and a real-time network intrusion detection system (NIDS). The NIDS controls the TM by dynamically adjusting recording parameters, instructing it to permanently store suspicious activity for offline forensics, and fetching traffic from the past for retrospective analysis. We present a detailed performance evaluation of both stand-alone and joint setups, and report on experiences with running the system live in high-volume environments.