Detection Of Zero-day Attacks Research Articles

ASAP: Automatic Synthesis of Attack Prototypes, an online-learning, end-to-end approach

Zero-day attack detection and categorization is an open-research field where four main context factors need to be taken into account: novel or zero-day attacks (i) are unlabeled by definition, (ii) may correspond to out-of-distribution data, (iii) can arise concurrently, and (iv) distribution shifts in the feature space need online-learning. Given such constraints, the online detection and categorization of new cyber threats can be modeled as a heterogeneous collective anomaly detection problem, for which no online-learning solutions exist purely based on back-propagation. To this respect, this paper presents an online-learning, end-to-end back-propagation strategy for Automatically Synthesizing the potential signatures or Attack Prototypes of novel cyber threats (asap). The presented framework incorporates automatic feature engineering, operating over raw data from the OpenFlow monitoring API and raw bytes of traffic captures. In asap, specialized inductive biases enhance the training data efficiency and accommodate the inference machinery to resource-constrained scenarios such as the Internet of Things. Finally, the validity of this framework is demonstrated in a live training experiment comprising IoT traffic emulation 33To foster research in the field, the source-code of asap alongside instructions to reproduce the experiments are made publicly available among the set of SmartVille NID models in [1]..

Contribution to Threat Management Through the Use of AI-Based IDS

Objectives: This paper aims to enhance cybersecurity through the integration of Artificial Intelligence (AI) in Intrusion Detection Systems (IDS), addressing the limitations of traditional IDS in detecting evolving cyber threats. Theoretical Framework: The study builds on existing research in cybersecurity, focusing on AI techniques such as decision trees and linear regression to improve the accuracy and effectiveness of AI-based IDS. Method: A comprehensive review of current AI-IDS methodologies is conducted, alongside an exploration of machine learning algorithms applied to datasets like KDD99 and NSL-KDD. The proposed architecture utilizes supervised machine learning to predict anomalies in network traffic. Results and Discussion: The findings indicate that AI-IDS can significantly reduce false positives and enhance detection of zero-day attacks through adaptive learning. The results highlight the importance of quality data and continuous model refinement. Research Implications: This research underscores the necessity for ongoing exploration of AI techniques in cybersecurity, suggesting future studies focus on real-time adaptive systems to further improve threat detection. Originality/Value: This paper contributes to the field by providing insights into the practical application of AI in IDS, offering a structured approach that combines theoretical knowledge with empirical evidence, thus paving the way for future innovations in cybersecurity.

Zero-X: A Blockchain-Enabled Open-Set Federated Learning Framework for Zero-Day Attack Detection in IoV

Zero-X: A Blockchain-Enabled Open-Set Federated Learning Framework for Zero-Day Attack Detection in IoV

Digital Twin and federated learning enabled cyberthreat detection system for IoT networks

The widespread deployment of Internet of Things (IoT) devices across various smart city applications presents significant security challenges, increased by the rapidly evolving landscape of cyber threats. Traditional security solutions, including those using Federated Learning with federated averaging, suffer from inefficiencies due to random node selection and partial data sampling, which can hinder the detection of comprehensive network-wide attacks. This paper introduces a novel Cyberthreat Detection System for IoT networks that leverages Digital Twin technology and an optimized Federated Learning approach. Our hypothesis implies integrating Digital Twin models within an IoT security framework to improve real-time cyberthreat detection capabilities. We implement a 'Adaptive Thresholding with Early Stopping method' based methodology in Federated Learning to systematically train and aggregate local models based on predefined training rounds, thereby ensuring that all local models contribute to the global model until a target accuracy is achieved. This method significantly improves the detection of zero-day attacks by reducing dependency on random selections and partial data. The system architecture features Digital Twins of IoT medical infrastructure components—such as radiology, intensive care, and outpatient care—positioned at the network edge to minimize latency and bandwidth usage. Comparative evaluations of our model against traditional federated averaging methods demonstrate superior performance, with enhancements in model aggregation efficiency evidenced by higher F1 scores and reduced CPU usage. Specifically, our distributed digital twin environment at the edge layer shows 14% and 33% latency reductions compared to fog and cloud-based implementations, respectively. This study highlights the potential of Digital Twin and advanced Federated Learning methodologies to secure IoT networks against evolving and growing cyber threats.

AI for AI-based intrusion detection as a service: Reinforcement learning to configure models, tasks, and capacities

Intrusion Detection Systems (IDS) increasingly leverage machine learning (ML) to enhance the detection of zero-day attacks. As operational complexities increase, enterprises are turning to Intrusion Detection as a Service (IDaS), requiring advanced solutions for efficient ML model selection and resource allocation. Existing research often focuses primarily on accuracy and computational efficiency, leaving a gap in solutions that can dynamically adapt. This study introduces a novel integrated solution, Auto-IDaS, which employs advanced Reinforcement Learning (RL) techniques for real-time, adaptive management of IDS. Auto-IDaS uses the Deep Q-Network (DQN) algorithm for dynamic ML model selection, automatically adjusting configurations of IDaS in response to fluctuating network traffic conditions. Simultaneously, it utilizes the Twin Delayed Deep Deterministic (TD3) algorithm for optimizing capacity allocation, aiming to minimize computational costs while maintaining service quality. This dual approach is innovative in its use of RL to address both selection and allocation challenges within IDaS frameworks. The effectiveness of TD3 is compared against Simulated Annealing (SA), a traditional optimization technique. The results demonstrate that utilizing DQN to dynamically select the model significantly improves the reward by 0.29% to 27.04%, effectively balancing detection performance (F1 score), detection time, and computation cost. Regarding capacity allocation, TD3 accelerates decision times approximately 5×106 times faster than SA while retaining decision quality within a 10% range comparable to SA’s performance.

NERO: NEural algorithmic reasoning for zeRO-day attack detection in the IoT: A hybrid approach

Anomaly detection approaches for network intrusion detection learn to identify deviations from normal behavior on a data-driven basis. However, current approaches strive to infer the degree of abnormality of out-of-distribution samples when these appertain to different zero-day attacks. Inspired by the successes of the neural algorithmic reasoning paradigm to leverage the generalization of rule-based behavior, this paper presents a deep learning strategy for solving zero-day network attack detection and categorization. Moreover, focusing on the particular scenario of the Internet of Things (IoT), the privacy preservation requirement may imply a low training data regime for any learning algorithm. To this respect, the presented framework uses metric-based meta-learning to achieve few-shot learning capabilities. The presented pipeline is called NERO, as it imports the encode-process-decode architecture from the NEural algorithmic reasoning blueprint to converge zeRO-day attack detection policies within constrained training data.

A framework for detecting zero-day exploits in network flows

Zero-day attack detection solutions aim to proactively identify unknown threats targeting valuable assets within a given system. While many Intrusion Detection System (IDS) solutions leverage learning techniques to build novel attack detection systems, they often focus on enhancing accuracy for specific attack types, overlooking the potential for multiple attack scenarios. Therefore, we introduce a novel framework for detecting zero-day attacks that evade current detection systems. Our framework enhances attack identification and qualification through a hybrid learning approach, where supervised learning ensures detection of known attacks and unsupervised learning. It encompasses intrusion detection phases from data collection to new attack class detection by identifying anomalies in real-time network flow data. Unsupervised learning, which involves grouping similar data points into clusters, establishes minimum distances within these clusters. This process triggers cluster division when certain thresholds are reached. Finally, an online supervised learning process validates our approach’s effectiveness in identifying anomalies associated with zero-day attack flows.This approach significantly reduces the False Detection Rate (FDR) without solely focusing on optimizing machine learning (ML) and deep learning (DL) algorithms hyper-parameters. We evaluated our framework on two datasets: one from a real industrial context at IBM and the NSL-KDD dataset. The results demonstrate our framework’s ability to detect anomalies in previously zero-day attack targets. On average, we identified 71 anomalous flows per target, achieving an overall average online learning accuracy of 98.4% for the IBM dataset and 96.6% for the NSL-KDD dataset, thereby validating the detection of these new attack scenarios.

Combining dynamic and static host intrusion detection features using variational long short-term memory recurrent autoencoder

Despite the many advantages offered by Host Intrusion Detection Systems (HIDS), they are rarely adopted in mainstream cybersecurity strategies. Unlike Network Intrusion Detection Systems, a HIDS is the last layer of defence between potential attacks and the underlying OSs. One of the main reasons behind this is its poor capabilities to adequately protect against zero-day attacks. With the rising number of zero-day exploits and related attacks, this is an increasingly imperative requirement for a modern HIDS. In this paper variational long short-term memory — recurrent autoencoder approach which improves zero-day attack detection is proposed. We have practically implemented our model using TensorFlow and evaluated its performance using benchmark ADFA-LD and UNM datasets. We have also compared the results against those from notable publications in the area.

Efficient Inductive Transfer Learning based Framework for Zero-Day Attack Detection

An Intrusion Detection System (IDS) is a type of security domain that tracks and evaluates network connections or system operations to detect potential security breaches, unauthorized usage, and malicious activity within computer networks. Machine learning (ML) and deep learning (DL) algorithms provide better IDS based on the labelled dataset. However, due to a lack of labelled data, its effectiveness in detecting zero-day attacks is limited. Anomaly detection methods frequently produce high False Positive Rates (FPR). Transfer learning (TL) is a powerful technique in various domains, including intrusion detection systems (IDS). It also creates advanced classifiers using knowledge extracted from the related source domain(s) with little or no labelled data. This paper introduced zero-day attack detection (ZDAD) model by combining it with transfer learning that helps classify the attacks and non-attacks from the given dataset. Using the UNSW-NB15 dataset, the authors created a Transfer Learning-based prototype in this study. The goal was to unify the feature space for distinguishing unlabeled Generic samples representing zero-day attacks from regular instances using labelled DoS samples. The ZDAD performed admirably, achieving 99.24% accuracy and a low False Positive Rate (FPR) of 0.02%. This performance outperforms current state-of-the-art methods.

From zero-shot machine learning to zero-day attack detection

AbstractMachine learning (ML) models have proved efficient in classifying data samples into their respective categories. The standard ML evaluation methodology assumes that test data samples are derived from pre-observed classes used in the training phase. However, in applications such as Network Intrusion Detection Systems (NIDSs), obtaining data samples of all attack classes to be observed is challenging. ML-based NIDSs face new attack traffic known as zero-day attacks that are not used in training due to their non-existence at the time. Therefore, this paper proposes a novel zero-shot learning methodology to evaluate the performance of ML-based NIDSs in recognising zero-day attack scenarios. In the attribute learning stage, the learning models map network data features to semantic attributes that distinguish between known attacks and benign behaviour. In the inference stage, the models construct the relationships between known and zero-day attacks to detect them as malicious. A new evaluation metric is defined as Zero-day Detection Rate (Z-DR) to measure the effectiveness of the learning model in detecting unknown attacks. The proposed framework is evaluated using two key ML models and two modern NIDS data sets. The results demonstrate that for certain zero-day attack groups discovered in this paper, ML-based NIDSs are ineffective in detecting them as malicious. Further analysis shows that attacks with a low Z-DR have a significantly distinct feature distribution and a higher Wasserstein Distance range than the other attack classes.

Zero-day attack detection: a systematic literature review

Zero-day attack detection: a systematic literature review

Survey on Intrusion Detection Systems Based on Machine Learning Techniques for the Protection of Critical Infrastructure.

Industrial control systems (ICSs), supervisory control and data acquisition (SCADA) systems, and distributed control systems (DCSs) are fundamental components of critical infrastructure (CI). CI supports the operation of transportation and health systems, electric and thermal plants, and water treatment facilities, among others. These infrastructures are not insulated anymore, and their connection to fourth industrial revolution technologies has expanded the attack surface. Thus, their protection has become a priority for national security. Cyber-attacks have become more sophisticated and criminals are able to surpass conventional security systems; therefore, attack detection has become a challenging area. Defensive technologies such as intrusion detection systems (IDSs) are a fundamental part of security systems to protect CI. IDSs have incorporated machine learning (ML) techniques that can deal with broader kinds of threats. Nevertheless, the detection of zero-day attacks and having technological resources to implement purposed solutions in the real world are concerns for CI operators. This survey aims to provide a compilation of the state of the art of IDSs that have used ML algorithms to protect CI. It also analyzes the security dataset used to train ML models. Finally, it presents some of the most relevant pieces of research on these topics that have been developed in the last five years.

Frequency Domain Feature Based Robust Malicious Traffic Detection

Machine learning (ML) based malicious traffic detection is an emerging security paradigm, particularly for zero-day attack detection, which is complementary to existing rule based detection. However, the existing ML based detection achieves low detection accuracy and low throughput incurred by inefficient traffic features extraction. Thus, they cannot detect attacks in realtime, especially in high throughput networks. Particularly, these detection systems similar to the existing rule based detection can be easily evaded by sophisticated attacks. To this end, we propose Whisper, a realtime ML based malicious traffic detection system that achieves both high accuracy and high throughput by utilizing frequency domain features. It utilizes sequential information represented by the frequency domain features to achieve bounded information loss, which ensures high detection accuracy, and meanwhile constrains the scale of features to achieve high detection throughput. In particular, attackers cannot easily interfere with the frequency domain features and thus Whisper is robust against various evasion attacks. Our experiments with 74 types of attacks demonstrate that, compared with the state-of-the-art systems, Whisper can accurately detect various sophisticated and stealthy attacks, achieving at most 18.36% improvement of AUC, while achieving two orders of magnitude throughput. Even under various evasion attacks, Whisper is still able to maintain around 90% detection accuracy.

Anomaly Detection of Zero-Day Attacks Based on CNN and Regularization Techniques

The rapid development of cyberattacks in the field of the Internet of things (IoT) introduces new security challenges regarding zero-day attacks. Intrusion-detection systems (IDS) are usually trained on specific attacks to protect the IoT application, but the attacks that are yet unknown for IDS (i.e., zero-day attacks) still represent challenges and concerns regarding users’ data privacy and security in those applications. Anomaly-detection methods usually depend on machine learning (ML)-based methods. Under the ML umbrella are classical ML-based methods, which are known to have low prediction quality and detection rates with regard to data that it has not yet been trained on. DL-based methods, especially convolutional neural networks (CNNs) with regularization methods, address this issue and give a better prediction quality with unknown data and avoid overfitting. In this paper, we evaluate and prove that the CNNs have a better ability to detect zero-day attacks, which are generated from nonbot attackers, compared to classical ML. We use classical ML, normal, and regularized CNN classifiers (L1, and L2 regularized). The training data consists of normal traffic data, and DDoS attack data, as it is the most common attack in the IoT. In order to give the full picture of this evaluation, the testing phase of those classifiers will include two scenarios, each having data with different attack distribution. One of these is the backdoor attack, and the other is the scanning attack. The results of the testing proves that the regularized CNN classifiers still perform better than the classical ML-based methods in detecting zero-day IoT attacks.

Composition of Hybrid Deep Learning Model and Feature Optimization for Intrusion Detection System

Recently, with the massive growth of IoT devices, the attack surfaces have also intensified. Thus, cybersecurity has become a critical component to protect organizational boundaries. In networks, Intrusion Detection Systems (IDSs) are employed to raise critical flags during network management. One aspect is malicious traffic identification, where zero-day attack detection is a critical problem of study. Current approaches are aligned towards deep learning (DL) methods for IDSs, but the success of the DL mechanism depends on the feature learning process, which is an open challenge. Thus, in this paper, the authors propose a technique which combines both CNN, and GRU, where different CNN-GRU combination sequences are presented to optimize the network parameters. In the simulation, the authors used the CICIDS-2017 benchmark dataset and used metrics such as precision, recall, False Positive Rate (FPR), True Positive Rate (TRP), and other aligned metrics. The results suggest a significant improvement, where many network attacks are detected with an accuracy of 98.73%, and an FPR rate of 0.075. We also performed a comparative analysis with other existing techniques, and the obtained results indicate the efficacy of the proposed IDS scheme in real cybersecurity setups.

When a RF beats a CNN and GRU, together—A comparison of deep learning and classical machine learning approaches for encrypted malware traffic classification

Internet traffic classification plays a crucial role in Quality of Experience (QoE), Quality of Services (QoS), intrusion detection, and traffic-trend analyses. While there is no theoretical guarantee that deep learning (DL)-based solutions perform better than classic machine learning (ML)-based ones, DL-based models have become the common default. This paper compares well-known DL-based and ML-based models and shows that in the case of malicious traffic classification, state-of-the-art DL-based solutions do not necessarily outperform the classical ML-based ones. We exemplify this finding using two well-known datasets for a varied set of tasks, such as: malware detection, malware family classification, detection of zero-day attacks, and classification of an iteratively growing dataset. Note that, it is not feasible to evaluate all possible models to make a concrete statement, thus the above finding is not a recommendation to avoid DL-based models, but rather an empirical finding that in some cases, there are more simplistic solutions, that may perform even better.

Transfer-Learning-Based Intrusion Detection Framework in IoT Networks

Cyberattacks in the Internet of Things (IoT) are growing exponentially, especially zero-day attacks mostly driven by security weaknesses on IoT networks. Traditional intrusion detection systems (IDSs) adopted machine learning (ML), especially deep Learning (DL), to improve the detection of cyberattacks. DL-based IDSs require balanced datasets with large amounts of labeled data; however, there is a lack of such large collections in IoT networks. This paper proposes an efficient intrusion detection framework based on transfer learning (TL), knowledge transfer, and model refinement, for the effective detection of zero-day attacks. The framework is tailored to 5G IoT scenarios with unbalanced and scarce labeled datasets. The TL model is based on convolutional neural networks (CNNs). The framework was evaluated to detect a wide range of zero-day attacks. To this end, three specialized datasets were created. Experimental results show that the proposed TL-based framework achieves high accuracy and low false prediction rate (FPR). The proposed solution has better detection rates for the different families of known and zero-day attacks than any previous DL-based IDS. These results demonstrate that TL is effective in the detection of cyberattacks in IoT environments.

SimCSE for Encrypted Traffic Detection and Zero-Day Attack Detection

Traffic detection has attracted much attention in recent years, playing an essential role in intrusion detection systems (IDS). This paper proposes a new approach for traffic detection at the packet level, inspired by natural language processing (NLP), using simple contrastive learning of sentence embeddings (SimCSE) as an embedding model. The new approach can learn the features of traffic from raw packet data. Experiments were conducted on two well-known datasets to evaluate our approach. For detecting malicious activity, our model achieved an accuracy of 99.99% on the USTC-TFC2016 dataset, whereas for detecting virtual private network (VPN) activity, our model achieved an accuracy of 99.98% on the ISCXVPN2016 dataset. Furthermore, the resulting model was found to be robust based on zero-day attack detection, which shows the model’s ability to detect attacks that have not been seen before. Experiments show that our approach can effectively detect network traffic and outperforms many other state-of-the-art methods.

Systematic Review of Graphical Visual Methods in Honeypot Attack Data Analysis

Mitigating increasing cyberattack incidents may require strategies such as reinforcing organizations’ networks with Honeypots and effectively analyzing attack traffic for detection of zero-day attacks and vulnerabilities. To effectively detect and mitigate cyberattacks, both computerized and visual analyses are typically required. However, most security analysts are not adequately trained in visualization principles and/or methods, which is required for effective visual perception of useful attack information hidden in attack data. Additionally, Honeypot has proven useful in cyberattack research, but no studies have comprehensively investigated visualization practices in the field. In this paper, we reviewed visualization practices and methods commonly used in the discovery and communication of attack patterns based on Honeypot network traffic data. Using the PRISMA methodology, we identified and screened 218 papers and evaluated only 37 papers having a high impact. Most Honeypot papers conducted summary statistics of Honeypot data based on static data metrics such as IP address, port, and packet size. They visually analyzed Honeypot attack data using simple graphical methods (such as line, bar, and pie charts) that tend to hide useful attack information. Furthermore, only a few papers conducted extended attack analysis, and commonly visualized attack data using scatter and linear plots. Papers rarely included simple yet sophisticated graphical methods, such as box plots and histograms, which allow for critical evaluation of analysis results. While a significant number of automated visualization tools have incorporated visualization standards by default, the construction of effective and expressive graphical methods for easy pattern discovery and explainable insights still requires applied knowledge and skill of visualization principles and tools, and occasionally, an interdisciplinary collaboration with peers. We, therefore, suggest the need, going forward, for non-classical graphical methods for visualizing attack patterns and communicating analysis results. We also recommend training investigators in visualization principles and standards for effective visual perception and presentation.

From Zero-Shot Machine Learning to Zero-Day Attack Detection

The standard ML methodology assumes that the test samples are derived from a set of pre-observed classes used in the training phase. Where the model extracts and learns useful patterns to detect new data samples belonging to the same data classes. However, in certain applications such as Network Intrusion Detection Systems, it is challenging to obtain data samples for all attack classes that the model will most likely observe in production. ML-based NIDSs face new attack traffic known as zero-day attacks, that are not used in the training of the learning models due to their non-existence at the time. In this paper, a zero-shot learning methodology has been proposed to evaluate the ML model performance in the detection of zero-day attack scenarios. In the attribute learning stage, the ML models map the network data features to distinguish semantic attributes from known attack (seen) classes. In the inference stage, the models are evaluated in the detection of zero-day attack (unseen) classes by constructing the relationships between known attacks and zero-day attacks. A new metric is defined as Zero-day Detection Rate, which measures the effectiveness of the learning model in the inference stage. The results demonstrate that while the majority of the attack classes do not represent significant risks to organisations adopting an ML-based NIDS in a zero-day attack scenario. However, for certain attack groups identified in this paper, such systems are not effective in applying the learnt attributes of attack behaviour to detect them as malicious. Further Analysis was conducted using the Wasserstein Distance technique to measure how different such attacks are from other attack types used in the training of the ML model. The results demonstrate that sophisticated attacks with a low zero-day detection rate have a significantly distinct feature distribution compared to the other attack classes.