Network Anomalies Research Articles

Machine Learning-Based Network Anomaly Detection: Design, Implementation, and Evaluation

Background: In the last decade, numerous methods have been proposed to define and detect outliers, particularly in complex environments like networks, where anomalies significantly deviate from normal patterns. Although defining a clear standard is challenging, anomaly detection systems have become essential for network administrators to efficiently identify and resolve irregularities. Methods: This study develops and evaluates a machine learning-based system for network anomaly detection, focusing on point anomalies within network traffic. It employs both unsupervised and supervised learning techniques, including change point detection, clustering, and classification models, to identify anomalies. SHAP values are utilized to enhance model interpretability. Results: Unsupervised models effectively captured temporal patterns, while supervised models, particularly Random Forest (94.3%), demonstrated high accuracy in classifying anomalies, closely approximating the actual anomaly rate. Conclusions: Experimental results indicate that the system can accurately predict network anomalies in advance. Congestion and packet loss were identified as key factors in anomaly detection. This study demonstrates the potential for real-world deployment of the anomaly detection system to validate its scalability.

Community detection and anomaly prediction in dynamic networks

Anomaly detection is an essential task in the analysis of dynamic networks, offering early warnings of abnormal behavior. We present a principled approach to detect anomalies in dynamic networks that integrates community structure as a foundational model for regular behavior. Our model identifies anomalies as irregular edges while capturing structural changes. Our approach leverages a Markovian framework for temporal transitions and latent variables for community and anomaly detection, inferring hidden parameters to detect unusual interactions. Evaluations on synthetic and real-world datasets show strong anomaly detection across various scenarios. In a case study on professional football player transfers, we detect patterns influenced by club wealth and country, as well as unexpected transactions both within and across community boundaries. This work provides a framework for adaptable anomaly detection, highlighting the value of integrating domain knowledge with data-driven techniques for improved interpretability and robustness in complex networks.

APPROACH TO DETECTION OF ANOMALOUS NETWORK TRAFFIC USING LOF AND HBOS ALGORITHMS

The article is devoted to the problem of detecting anomalies in modern computer networks, which is one of the main threats to cyber security. With the development of Internet technologies, the number of devices and the volume of network traffic are constantly increasing, which leads to an increase in the risk of various cyber threats, such as DDoS attacks, zero-day attacks, and exploitation of protocol vulnerabilities. Abnormal network traffic can result from malicious activity and technical malfunctions, such as configuration errors or hardware failures. Specialised algorithms and methods of analysing large volumes of data are used to detect such threats. The paper considers the main methods of detecting anomalies in network traffic, including classical approaches and modern deep and machine learning methods. Special attention is paid to the efficiency of using methods based on convolutional neural networks, long-term memory and their combinations to detect anomalies. An analysis of the disadvantages and advantages of various approaches to detecting anomalous traffic, such as high computational requirements and the complexity of setting up models, is performed. Still, their effectiveness in analysing large volumes of data is noted. One of the main methods used for anomaly analysis is the local outlier algorithm, which compares the density of objects with their neighbours, allowing for the detection of anomalies in regional segments of the data. Another method is histogram-based outlier estimation, which is faster and more efficient using one-dimensional histograms for each variable. The work also explores the application of unsupervised machine learning methods, which allows for analysing network traffic in real time without the need for prior labelling of data. The article also considers the prospects of further testing the proposed methods in real networks. The combined use of LOF and HBOS balances anomaly detection accuracy and data processing speed, essential to ensure continuous system operation in high-load networks. The implementation of similar solutions in actual conditions requires further research, particularly regarding optimising the use of computing resources and adapting methods to the specific conditions of the network environment. Thus, the paper presents a thorough analysis of modern approaches to detecting anomalies in network traffic and substantiates the feasibility of their application in actual conditions to increase the effectiveness of cyber security.

Machine Learning-Based Methodologies for Cyber-Attacks and Network Traffic Monitoring: A Review and Insights

The number of connected IoT devices is increasing significantly due to their many benefits, including automation, improved efficiency and quality of life, and reducing waste. However, these devices have several vulnerabilities that have led to the rapid growth in the number of attacks. Therefore, several machine learning-based intrusion detection system (IDS) tools have been developed to detect intrusions and suspicious activity to and from a host (HIDS—Host IDS) or, in general, within the traffic of a network (NIDS—Network IDS). The proposed work performs a comparative analysis and an ablative study among recent machine learning-based NIDSs to develop a benchmark of the different proposed strategies. The proposed work compares both shallow learning algorithms, such as decision trees, random forests, Naïve Bayes, logistic regression, XGBoost, and support vector machines, and deep learning algorithms, such as DNNs, CNNs, and LSTM, whose approach is relatively new in the literature. Also, the ensembles are tested. The algorithms are evaluated on the KDD-99, NSL-KDD, UNSW-NB15, IoT-23, and UNB-CIC IoT 2023 datasets. The results show that the NIDS tools based on deep learning approaches achieve better performance in detecting network anomalies than shallow learning approaches, and ensembles outperform all the other models.

Detection of Traffic Anomalies Based on Their Frame Wavelet Transformations Processing

Relevance. The active transition to a massive digital infrastructure based on Internet of Things (IoT) technology has brought telecommunications networks to the level of dominant information resources. The one-time increase in the number of existing Internet services is inextricably linked to the growing variety of network anomalies on telecommunications equipment. In turn, existing methods of detecting network threats do not allow timely assessment of network traffic, which is characterized by a large number of parameters, and the detected anomalies from external interference do not have pronounced patterns. The purpose of the study is to increase the efficiency of detecting traffic anomalies based on the results of processing its frame wavelet transform. The scientific task is to develop scientific and methodological approaches that allow effective analysis and timely detection of anomalies in network traffic. A comparative review of search methods for detecting network traffic anomalies, algorithms for detecting uncontrolled anomalies, traffic analysis methods based on local emission factor, binary trees, optical emission spectroscopy. Decision. The results of the study of the possibility of detecting anomalies in the bitstream traffic based on the results of its multiple-variable transformation in the Haar wavelet basis are considered. The choice for further processing of the coefficients of the traffic decomposition matrix along the time shift variable is justified. It is proved that multiple-scale transformations not only increase the structural differences in traffic, but also open up the possibility of localization of anomalies that caused these differences. The scientific novelty of the work is determined by the author's approach to detecting network traffic anomalies during the transition from the direct representation of a signal in the form of its discrete samples to coefficients formed from the matrices of its wavelet transformations, and, as a result, increasing its contrast with other signals with a similar structure. Theoretical significance. The necessity and sufficiency of using wavelet coefficients instead of time samples of signals in the basis of the parent wavelet from the matrix of the generated frame is proved. The relationship between the Hurst indicators and the coefficients of the cross-correlation functions has been established. Practical significance. The results obtained in the work, in the future, can be used in the construction of models for evaluating network traffic in conditions of deliberate, as well as methods for searching and synthesizing effective methods of protection against them.

Detection of Incidents and Anomalies in Software-Defined Network – Based Implementations of Critical Infrastructure Resulting in Adaptive System Changes

Detection of Incidents and Anomalies in Software-Defined Network – Based Implementations of Critical Infrastructure Resulting in Adaptive System Changes

Designing a Novel Network Anomaly Detection Framework using Multi-Serial Stacked Network with Optimal Feature Selection Procedures over DDOS Attacks

- Distributed denial-of-service (DDoS) attacks are the major threat that disrupts the services in the computer system and networks using traffic and targeted sources. So, real-world attack detection techniques are considered an important element in executing cybersecurity tasks. The present DDoS techniques are prone to False Positive Rates (FPR) and also it didn’t acquire the complicated patterns presented in the attack traffic. Internet of Things (IoT) is a complicated network with resource-constrained devices and networks that are prone to different security threats like DDoS attacks. Later, the Software Defined Networking (SDN) with IoT models is used to enhance the access control techniques and security models. DDoS attacks are considered as an important threat in the IoT networks. Hence, it is important to construct a novel network anomaly detection model with a deep learning mechanism to resolve the limitations of the existing techniques. Initially, essential data required for the validation are gathered from the IDS ISCX 2012 dataset. The optimal features are selected from input data using the Predefined-Mud Ring Algorithm (P-MRA). The optimally selected features are provided to the Multi-Serial Stacked Networks (Multi-SSN), which is the fusion of Convolutional Autoencoder (CAE), Gated Recurrent Unit (GRU), and Bayesian Learning (BL) networks. Here, the essential features for the validation are acquired from the CAE and GRU. Then, these features are stacked and given to the BL mechanism for detecting the anomalies in the network. Further, several experimental validations are performed in the developed framework over traditional network anomaly detection mechanism.

Improved identification of network anomalies through optimal CURE clustering

Abstract In this paper, we propose an advanced network anomaly behavior identification framework to overcome the constraints inherent in conventional rule- or signature-based approaches, which often struggle with emerging and previously unknown threats. Central to our framework is an Enhanced CURE (Cluster Updating and REfining) clustering algorithm, meticulously tailored and refined to incorporate a density-based methodology. This enhancement enables the algorithm to discern subtle shifts in network anomaly patterns with heightened precision. The implementation workflow commences with the application of the optimized CURE algorithm to analyze network data, followed by the deployment of a sophisticated anomaly degree ranking mechanism. This mechanism, through meticulous calculation of individual data points’ anomaly degrees and subsequent ranking, effectively isolates those deviating significantly from standard behavioral norms, incorporating a strategic threshold to filter out false positives. To validate our methodology’s efficacy and its superiority over existing techniques, experiments were conducted utilizing a substantial real-world network dataset. These tests affirm not only a marked increase in the accuracy of abnormal behavior recognition and a reduction in computational intricacy but also demonstrate the adaptability across diverse network ecosystems. Our approach has proven successful in pinpointing a wide array of network anomalies, encompassing malicious cyberattacks, fraudulent activities, unauthorized intrusions, and breaches of security protocols, thereby highlighting its comprehensive capability in bolstering network defense strategies. Despite the notable advancements and successful identification of various network anomalies, our framework currently lacks integration with real-time learning capabilities, limiting its immediate responsiveness to rapidly evolving attack patterns and necessitating ongoing research for dynamic updates and adaptive learning mechanisms.

Taxonomy and Survey of Collaborative Intrusion Detection System using Federated Learning

This review paper looks at recent research on Federated Learning (FL) for Collaborative Intrusion Detection Systems (CIDS) to establish a taxonomy and survey. The motivation behind this review comes from the difficulty of detecting coordinated cyberattacks in large-scale distributed networks. Collaborative anomalies are one of the network anomalies that need to be detected through robust collaborative learning methods. FL is promising collaborative learning method in recent research. This review aims to offer insights and lesson learn for creating a taxonomy of collaborative anomaly detection in CIDS using FL as a collaborative learning method. Our findings suggest that a taxonomy is required to map the discussion area, including an algorithm for training the learning model, the dataset, global aggregation model, system architecture, security, and privacy. Our results indicate that FL is a promising approach for collaborative anomaly detection in CIDS, and the proposed taxonomy could be useful for future research in this area. Overall, this review contributes to the growing knowledge of FL for CIDS, providing insights and lessons for researchers and practitioners. This research also concludes significant challenges, opportunities, and future directions in CIDS based on collaborative anomaly detection using FL.

Modelling and monitoring multi-relational networks with ordinal information

Network relationships can be widely seen among entities in various fields such as social networks, supply networks and Internet of Things (IoT). Sometimes abnormal events such as cyber-attacks occur to cause an abrupt increase or decrease in the traffic of networks. Many anomaly detection methods have been developed to identify such abnormal events in networks. In recent years, statistical process control (SPC) has attracted more and more attention in network anomaly detection. However, many of the existing statistical models regard the interaction between two nodes in unweighted directed networks as a binary variable, i.e. presence and absence of contacts, which fails to reflect the intensity level of interactions. This article proposes a new model to describe the dyadic interactions with several ordinal levels and introduces special quantities to incorporate the ordinal information into the model. The model can be expressed in a matrix form to enable easy parameter estimation and derivation of a quadratic monitoring statistic. Numerous simulation studies show that the proposed methods detect anomalies in multi-relational networks more quickly than existing monitoring methods. A case study exhibits the implementation and superiority of the proposed method.

Enhancing the security in IoT and IIoT networks: An intrusion detection scheme leveraging deep transfer learning

The Internet of Things (IoT) networks, which are defined by their interconnected devices and data streams are an expanding attack surface for cyber adversaries. Industrial Internet of Things (IIoT) is a subset of IoT and has significant importance in-terms of security. Robust intrusion detection systems (IDS) are essential for protecting these critical infrastructures. Our research suggests a novel approach to the detection of anomalies in IoT and IIoT networks that leverages the capabilities of deep transfer learning. Our methodology begins with the EdgeIIoT dataset, which serves as the basis for our data analysis. We convert the data into an appropriate image format to enable Convolutional Neural Network (CNN)-based processing. The hyper-parameters of individual machine learning models are subsequently optimized using a Random Search algorithm. This optimization phase optimizes the performance of each model by modifying the hyper-parameters that are unique to the learning algorithms. The performance of each model is meticulously assessed subsequent to hyper-parameter optimization. The top-performing models are subsequently, strategically selected and combined using the ensemble technique. The IDS scheme’s overall detection accuracy and generalizability are improved by the integration of strengths from multiple models. The proposed scheme demonstrates significant effectiveness in identifying a broad spectrum of attacks, encompassing a total of 14 distinct attack types. This comprehensive detection capability contributes to a more secure and resilient IoT ecosystem. Furthermore, application of quantization to our best models reduces resource utilization significantly without compromising accuracy.

Abnormal Structural–Functional Coupling and MRI Alterations of Brain Network Topology in Progressive Supranuclear Palsy

BackgroundProgressive supranuclear palsy (PSP) can cause structural and functional brain reconstruction. There is a lack of knowledge about the consistency between structural–functional (S–F) connection networks in PSP, despite growing evidence of anomalies in various single brain network parameters.PurposeTo study the changes in the structural and functional networks of PSP, network's topological properties including degree, and the consistency of S–F coupling. The relationship with clinical scales was examined including the assessment of PSP severity, and so on.Study TypeRetrospective.SubjectsA total of 51 PSP patients (70.04 ± 7.46, 25 females) and 101 healthy controls (64.58 ± 8.84, 58 females).Field Strength/Sequence3‐T, resting‐state functional MRI, diffusion tensor imaging, and T1‐weighted images.AssessmentA graph‐theoretic approach was used to evaluate structural and functional network topology metrics. We used the S–F coupling changes to explore the consistency of structural and functional networks.Statistical TestsIndependent samples t tests were employed for continuous variables, χ2 tests were used for categorical variables. For network analysis, two‐sample t tests was used and implied an false discovery rate (FDR) correction. Pearson correlation analysis was used to explore the correlations. A P‐value <0.05 was considered statistically significant.ResultsPSP showed variations within and between modules. Specifically, PSP had decreased network properties changes (t = −2.0136; t = 2.5409; t = −2.5338; t = −2.4296; t = −2.5338; t = 2.8079). PSP showed a lower coupling in the thalamus and left putamen and a higher coupling in the visual, somatomotor, dorsal attention, and ventral attention network. S–F coupling was related to the number of network connections (r = 0.32, r = 0.22) and information transmission efficiency (r = 0.55, r = 0.28). S–F coupling was related to basic academic ability (r = 0.39) and disinhibition (r = 0.49).Data ConclusionPSP may show abnormal S–F coupling and intramodular and intermodular connectome in the structural and functional networks.Level of Evidence3Technical EfficacyStage 3

Robust AI Based Bio Inspired Protocol using GANs for Secure and Efficient Data Transmission in IoT to Minimize Data Loss

Objectives: To propose an AI-based protocol to enhance the reliability and security of IoT data transmission in a robust manner. Deep learning with bio-inspired methods is employed to address real-time challenges such as route optimization, data integrity, error recovery, and end-to-end delay in order to minimize data loss and maximize the transmission rate. Methods: Generative Adversarial Networks (GANs) are used to enhance the robustness of the protocol in combination with a bio-inspired Artificial Immune System (AIS) to detect IoT network anomalies and responds to malicious activities by using the adaptive learning capabilities of IS. Hybrid Automatic Repeat Request (HARQ), an error recovery method, is utilized to detect network errors in order to correct and retransmit data to the destination during error-prone network conditions. Queue learning action sets are used to discover the finest path for efficient data transmission. The OMNeT++ simulator is used to assess the performance of the proposed BIP-GANs protocol. The performance results are compared with the prevailing data transmission protocols, such as EAP-IFBA, DNN-CSO, and ALO-DHT. Findings: The suggested BIP-GANs data transmission protocol outperforms with promising results, with an energy consumption rate of 6%, 8 seconds data transmission speed, 6 second less delay rate, 98% robustness (security and confidentiality), 98.5% alive nodes, and 99% network life span, which is higher than the prevailing methods. Novelty: This research study provides a comprehensive solution to secure and efficient data transmission by integrating the AI-based bio-inspired BIP-GANs method to address the security and data transmission challenges of existing methods such as EAP-IFBA, DNN-CSO, and ALO-DHT in terms of energy consumption rate, transmission speed, delay rate, life span of the network, robustness, and number of alive nodes. Keywords: Artificial Intelligence, Generative Adversarial Networks, Error Recovery, Secured Data Transmission, Deep Learning, Bio­Inspired Algorithm

Simulation of Artificial Intelligence Algorithm Based on Network Anomaly Detection and Wireless Sensor Network in Sports Cardiopulmonary Monitoring System

Simulation of Artificial Intelligence Algorithm Based on Network Anomaly Detection and Wireless Sensor Network in Sports Cardiopulmonary Monitoring System

Campus Network Traffic Prediction and Anomaly Detection Based on Deep Learning

This paper proposes an intelligent solution for network traffic prediction and anomaly detection in campus networks, addressing the increasingly severe network security challenges. The proposed approach innovatively integrates Convolutional Neural Networks (CNN) and Long Short-Term Memory networks (LSTM) to simultaneously extract local features and capture dynamic temporal dependencies of network traffic, significantly improving prediction accuracy. Based on this, an adaptive threshold anomaly detection algorithm is designed to automatically adjust detection sensitivity according to traffic variations, achieving a better balance between accuracy and recall rates. Additionally, an anomaly visualization scheme is presented, intuitively displaying the spatiotemporal distribution of network anomalies through heatmaps, assisting administrators in decision-making. Large-scale experiments demonstrate that this approach can effectively identify various security threats such as DDoS attacks, scanning probes, and botnets, with an overall detection rate exceeding 90% while maintaining a low false positive rate. Compared to traditional statistical and machine learning methods, the proposed approach exhibits stronger adaptability and generalization capabilities, providing crucial support for building an intelligent, precise, and reliable campus network security protection system. Future work will focus on further improving the real-time performance and robustness of the solution, expanding its application in new network scenarios such as IoT and edge computing.

Towards intrusion detection in fog environments using generative adversarial network and long short-term memory network

Recently, fog computing has been developed to complement cloud computing, which can provide cloud services at the edge of the network with real-time processing. However, the computational power of fog nodes is limited and this leads to security issues. On the other hand, cyber-attacks have become common with the exponential growth of Internet of Things (IoT) connected devices. This fact necessitates the development of Intrusion Detection Systems (IDSs) in fog environments with the aim of detecting attacks. In this paper, we develop an IDS named GAN-LSTM for fog environments that uses Generative Adversarial Networks (GANs) and Long Short-Term Memory Networks (LSTMs). GAN-LSTM is used to identify anomalies in network traffic to specific types of attacks or non-attacks. In general, GAN-LSTM consists of three components: data preprocessing, generation of real traffic patterns, and sequence analysis of real traffic data. Data preprocessing ensures data quality by removing noise and irrelevant features. The pre-processed data is fed to the GAN to generate real traffic as a baseline for normal behavior. Finally, the LSTM component is applied to detect anomalous anomalies in fog computing. The proposed algorithm was evaluated on public databases and experimental results showed that GAN-LSTM improves the accuracy of attack detection compared to equivalent approaches.

Corporate network anomaly detection methodology utilizing machine learning algorithms

ABSTRACT This study addresses the critical need for securing corporate networks against anomalies, a pressing concern in ensuring the comprehensive security of these networks. It aims to develop and validate a new machine learning-based methodology for anomaly detection that is adaptable across various corporate network environments, highlighting the method’s potential practical applications. Employing a systematic approach, the research integrates system analysis of anomaly detection methodologies with an analytical review of machine learning techniques tailored for high-security measures and attack prevention in corporate networks. This dual approach ensures a robust framework for identifying and addressing network anomalies efficiently. The methodology demonstrated notable efficacy, with the proposed machine learning-based anomaly detection techniques achieving an efficiency rate upwards of 90% in identifying and categorizing network traffic types. This high level of precision allows for the effective tracking of network anomalies across diverse corporate networks and their respective devices and equipment. The findings underscore the substantial practical value of the developed methodology, offering a promising avenue for enhancing corporate network security. The implementation of this machine learning-based approach not only facilitates the timely detection of anomalies but also significantly contributes to the improvement of machine learning applications within the realm of network security. Future research could further refine these techniques, exploring scalability and real-time data analysis enhancements to bolster their effectiveness across various network configurations.

Anomaly detection in network traffic with ELSC learning algorithm

AbstractIn recent years, the internet has not only enhanced the quality of our lives but also made us susceptible to high‐frequency cyber‐attacks on communication networks. Detecting such attacks on network traffic is made possible by intrusion detection systems (IDS). IDSs can be broadly divided into two groups based on the type of detection they provide. According to the established rules, the first signature‐based IDS detects threats. Secondly, anomaly‐based IDS detects abnormal conditions in the network. Various machine and deep learning approaches have been used to detect anomalies in network traffic in the past. To improve the detection of anomalies in network traffic, researchers have compared several machine learning models, such as support vector machines (SVM), logistic regressions (LRs), K‐Nearest Neighbour (KNN), Nave Bayes (NBs), and boosting algorithms. The accuracy, precision, and recall of many studies have been satisfactory to an extent. Therefore, this paper proposes an ensemble learning‐based stacking classifier (ELSC) to achieve a better accuracy rate. In the proposed ELSC algorithm, KNN, NB, LR, and Decision Trees (DT) served as the base classifiers, while SVM served as the meta classifier. Based on a Network Intrusion detection dataset provided by Kaggle.com, ELSC is compared to base classifiers such as KNN, NB, LR, DT, SVM, and Linear Discriminate Analysis. As a result of the simulations, the proposed ELBS stacking classifier was found to outperform the other comparative models and converge with an accuracy of 99.4%.

An Identification Method for Anomaly Types of Active Distribution Network Based on Data Mining

With the increasing penetration of distributed generators (DGs) and the growing demand for reliable power sources, it has become imperative to promptly identify anomalies in active distribution networks (ADNs). Additionally, anomaly identification is also crucial to assisting in targeting maintenance to handle failures after the action of relay protections. This study presents a highly accurate and rapidly responsive approach to identifying anomalies in ADNs. The proposed method uses three layers of intricately interconnected modules. In the first module, an iterative clustering layer is devised to process the original data. In the second module, an association rule layer is designed to identify hidden patterns based on the types of anomalies and the post-fault voltage. In the third module, a regression training layer is employed to train accurate identification models. The case study utilizes data samples collected by an IEEE 33-bus system with distributed generations as well as post-fault data from a practical distribution network. The results of comparative tests demonstrate that the proposed data-mining structure achieves reliable performance with high identification accuracy and short reaction times, indicating its ability to be applied to the state awareness system of ADNs.

Machine learning-based anomaly detection for smart home networks under adversarial attack

As smart home networks become more widespread and complex, they are capable of providing users with a wide range of applications and services. At the same time, the networks are also vulnerable to attack from malicious adversaries who can take advantage of the weaknesses in the network's devices and protocols. Detection of anomalies is an effective way to identify and mitigate these attacks; however, it requires a high degree of accuracy and reliability. This paper proposes an anomaly detection method based on machine learning (ML) that can provide a robust and reliable solution for the detection of anomalies in smart home networks under adversarial attack. The proposed method uses network traffic data of the UNSW-NB15 and IoT-23 datasets to extract relevant features and trains a supervised classifier to differentiate between normal and abnormal behaviors. To assess the performance and reliability of the proposed method, four types of adversarial attack methods: evasion, poisoning, exploration, and exploitation are implemented. The results of extensive experiments demonstrate that the proposed method is highly accurate and reliable in detecting anomalies, as well as being resilient to a variety of types of attacks with average accuracy of 97.5% and recall of 96%.