Network Anomalies Research Articles

Traditional Methods and Machine Learning for Anomaly Detection in Self-Organizing Networks

The motivation behind exploring anomaly detection in self-organizing networks lies in the evolving landscape of telecommunications and network management. Conventional methods for identifying network anomalies often struggle to adapt to the dynamic and complex nature of modern self-organizing networks. The problem addressed in this research is the efficacy of anomaly detection methods in self-organizing networks (SONs) within the context of telecommunications and network management. As SONs become increasingly prevalent to meet the demands of modern, highly dynamic wireless communication systems, the need for robust anomaly detection mechanisms is paramount. Conventional anomaly detection approaches in SONs are often based on predefined rules and thresholds, which may struggle to adapt to the intricate and rapidly evolving network behaviors. These methods can result in false alarms, missed anomalies, and inefficient resource allocation. Furthermore, emerging SONs incorporate a multitude of diverse technologies, including 5G, IoT, and edge computing, compounding the complexity of anomaly detection. Contemporary machine learning techniques hold promise in addressing these challenges by enabling the automatic and adaptive detection of anomalies, leveraging the abundance of data generated in SONs. However, the suitability, performance, and scalability of these methods in dynamic and large-scale SON environments remain critical concerns. This research aims to compare and evaluate conventional anomaly detection methods against contemporary machine learning approaches in SONs to assess their accuracy, efficiency, and adaptability. The goal is to provide insights into the most effective anomaly detection strategies, ultimately enhancing network stability, minimizing downtime, and ensuring the secure and efficient operation of modern telecommunications systems.

Graph neural network based robust anomaly detection at service level in SDN driven microservice system

Software-Defined Network (SDN) has been widely employed in data center networks to host various software systems such as microservice systems. A microservice system is constructed by a microservice architecture composed of dozens of services running in different lightweight virtual machines (e.g., containers) and relying on overlay networks for connectivity. However, with the increase of the system scale and the network traffic volume, the network becomes more complex and common to behave anomalously. Therefore, it is error-prone to manually investigate anomalies. To tackle this problem, we propose a non-intrusive monitoring system to help network operators obtain deep insights from the network traffic generated by the running system. Based on the monitoring system, we propose an anomaly detection method to automatically detect network anomalies. The core idea is to learn behavior patterns of inter/intra-components by modeling spatial–temporal relationships amongst collected network metrics with the proposed model STCell-VAE. The pattern will be leveraged to reconstruct the input data. Subsequently, the reconstruction probability is used to determine anomalies. Additionally, STCell-VAE can assimilate network information from new components as the network evolves, eliminating the necessity for retraining. Experiments are conducted within a real networking system and four simulated large-scale networks. The results demonstrate that our system can detect anomalies with about 0.7 F1-score, which outperforms state-of-the-art methods.

ILIDViz: An Incremental Learning-Based Visual Analysis System for Network Anomaly Detection

BackgroundWith the development of information technology, network traffic logs mixed with various kinds of cyber-attacks have grown explosively. Traditional intrusion detection systems (IDS) have limited ability to discover new inconstant patterns and identify malicious traffic traces in real-time. It is urgent to implement more effective intrusion detection technologies to protect computer security. MethodsIn this paper, we design a hybrid IDS, combining our incremental learning model (KAN-SOINN) and active learning, to learn new log patterns and detect various network anomalies in real-time. Results & ConclusionsThe experimental results on the NSLKDD dataset show that the KAN-SOINN can be improved continuously and detect malicious logs more effectively. Meanwhile, the comparative experiments prove that using a hybrid query strategy in active learning can improve the model learning efficiency.

Abnormal line loss identification of distributed PV low-voltage distribution area based on data driven

The identification of line loss anomalies in low-voltage distribution networks has been a very challenging problem for a long time. The large-scale access of distributed photovoltaics to the distribution network changes the power flow distribution and further increases the difficulty of identifying abnormal line losses in low-voltage distribution areas. In this paper, an identification method for abnormal line loss in distributed photovoltaic access low-voltage distribution area is proposed. Firstly, we analyze the correlation between the line loss rate and other indicators in the area of distributed photovoltaic access. Secondly, according to the gray correlation degree results, the appropriate indicators are selected, and the k-means clustering algorithm is used to cluster these indicators, through the clustering results, we can detect outliers to determine whether there is a line loss anomaly in the area. Finally, by analyzing the time dispersion degree of the cluster where the outlier point is located, the anomaly coefficient of the area is obtained, and the line loss anomaly is judged according to the anomaly coefficient. To verify the effectiveness of this method, we conduct an empirical analysis of typical containing distributed photovoltaics. Experimental results show that this method can effectively identify the line loss abnormalities in the distribution area containing distributed photovoltaics.

Adaptive Label Propagation for Group Anomaly Detection in Large-Scale Networks

This paper concentrates on group anomalies in general large-scale networks. Existing algorithms on group anomalies mainly focus on homogeneous or bipartite networks, and thus are difficult to apply to heterogeneous networks directly. Moreover, these algorithms follow the non-overlapping hypothesis of groups implicitly, which is improper in many scenarios. In this paper, we introduce a novel algorithm called Adaptive Label Propagation (ALP) to solve these problems. ALP is designed based on label propagation (LP) frameworks, for the reason that LP-based frameworks are simple in thought and easy to scale. ALP is able to find overlapping groups by label propagation with belonging coefficients, and can be applied to heterogeneous networks for its design of adaptive neighbor weighting. Assigning different weights to neighbors in label propagation is a challenging task. Inspired by the combinatorial multi-armed bandit mechanism, ALP views the neighbors of each node as arms to be selected, and iteratively updates their weights by evaluating their expected rewards in following iterations. Experiments are conducted on four real-world networks. The results show that LP-based methods are effective for detecting group anomalies, and the comparison results with several state-of-the-art label propagation based community detection methods show the effectiveness of the proposed method

Optimizing the Performance of Network Anomaly Detection Using Bidirectional Long Short-Term Memory (Bi-LSTM) and Over-sampling for Imbalance Network Traffic Data

Optimizing the Performance of Network Anomaly Detection Using Bidirectional Long Short-Term Memory (Bi-LSTM) and Over-sampling for Imbalance Network Traffic Data

SMDFbs: Specification-Based Misbehavior Detection for False Base Stations

The advancement of cellular communication technology has profoundly transformed human life. People can now watch high-definition videos anytime, anywhere, and aim for the implementation of advanced autonomous driving capabilities. However, the sustainability of such an environment is threatened by false base stations. False base stations execute attacks in the Radio Access Network (RAN) of cellular systems, adversely affecting the network or its users. To address this challenge, we propose a behavior rule specification-based false base station detection system, SMDFbs. We derive behavior rules from the normal operations of base stations and convert these rules into a state machine. Based on this state machine, we detect network anomalies and mitigate threats. We conducted experiments detecting false base stations in a 5G RAN simulator, comparing our system with seven machine learning-based detection techniques. The experimental results showed that our proposed system achieved a detection accuracy of 98% and demonstrated lower overhead compared to other algorithms.

Unleashing the power of Bat optimized CNN-BiLSTM model for advanced network anomaly detection: Enhancing security and performance in IoT environments

The growth of IoT (Internet of Things) devices has revolutionized several industries and brought about novel security threats. Recognizing network anomalies that may point to malicious activity or system flaws is a major issue. Traditional anomalous identification methods frequently need to catch up when dealing with the special traits of IoT environments, including resource limitations and changing network behavior. This paper introduces an innovative approach, the Bat-optimized CNN-BiLSTM model, to enhance the security and efficiency of IoT environments. This model combines the strengths of Convolutional Neural Networks (CNNs) for spatial analysis and Bidirectional Long Short-Term Memory (BiLSTM) networks for capturing temporal patterns, thus effectively representing time and space trends in IoT data. To optimize its performance further, researchers have leveraged the Bat algorithm, inspired by natural behaviors, to fine-tune the model. This program effectively searches for the best network anomaly detection parameters by imitating the echo activity of bats. Researchers want to increase detection accuracy by lowering false positives and false negatives using the Bat algorithm to enhance the CNN-BiLSTM model. The experimental findings show that the Bat-optimised CNN-BiLSTM model beats the state-of-the-art anomaly detection methods with 99.43% accuracy and efficiency.

A hybrid methodology for anomaly detection in Cyber–Physical Systems

The rapid adoption of Industry 4.0 has seen Information Technology (IT) networks increasingly merged with Operational Technology (OT) networks, which have traditionally been isolated on air-gapped and fully trusted networks. This increased attack surface has resulted in compromises of Cyber–Physical Systems (CPS) with significant economic and life safety consequences. This paper proposes a hybrid model of anomaly detection of security threats to CPS by blending the signature-based and threshold-based Intrusion Detection Systems (IDS) commonly used in IT networks, with a Machine Learning (ML) model designed to detect behaviour-based anomalies in OT networks. This hybrid model achieves more rapid detection of known threats through signature-based and threshold-based detection strategies, and more accurate detection of unknown threats via behaviour-based anomaly detection using ML algorithms.

Detection of Unauthorized Actions in Networks Using Wavelet Analysis

Signal processing techniques are used to analyze and detect network anomalies because of their ability to detect new and unknown intrusions. The paper proposes a method of modeling network signals for the detection of network anomalies, which combines wavelet approximation and the theory of system identification. To characterize the behavior of network traffic, fifteen functions are provided, which are used as input signals within the system. At the same time, it is assumed that security violations within the network can be detected by checking abnormal patterns of system functioning according to audit data.
 Despite the fact that machine learning methods have achieved significant results in detecting network anomalies, they still face the difficulty of using the implemented algorithms, in the presence of differences in the behavior of the training data and test data, which in turn leads to inefficient performance of the algorithms. This effect is exacerbated by the limitation of algorithms to detect previously unknown types of attacks due to the large number of false positives.
 The paper develops a new method of modeling network signals for detecting anomalies in networks using wavelet analysis. In particular, the general architecture of the approach consists of three components: feature analysis, modeling of normal network traffic based on wavelet approximation and prediction using ARX model, intrusion or non-intrusion decision making
 The result is evaluated using the DARPA intrusion detection dataset, which performs a comprehensive analysis of the intrusions in the dataset. Evaluation results show that this approach provides a high level of detection of both instances and types of attacks.

Support vector data description with kernel density estimation (SVDD-KDE) control chart for network intrusion monitoring

Multivariate control charts have been applied in many sectors. One of the sectors that employ this method is network intrusion detection. However, the issue arises when the conventional control chart faces difficulty monitoring the network-traffic data that do not follow a normal distribution as required. Consequently, more false alarms will be found when inspecting network traffic data. To settle this problem, support vector data description (SVDD) is suggested. The control chart based on the SVDD distance can be applied for the non-normal distribution, even the unknown distributions. Kernel density estimation (KDE) is the nonparametric approach that can be applied in estimating the control limit of the non-parametric control charts. Based on these facts, a multivariate chart based on the integrated SVDD and KDE (SVDD-KDE) is proposed to monitor the network's anomaly. Simulation using the synthetic dataset is performed to examine the performance of the SVDD-KDE chart in detecting multivariate data shifts and outliers. Based on the simulation results, the proposed method produces better performance in detecting shifts and higher accuracy in detecting outliers. Further, the proposed method is applied in the intrusion detection system (IDS) to monitor network attacks. The NSL-KDD data is analyzed as the benchmark dataset. A comparison between the SVDD-KDE chart with the other IDS-based-control chart and the machine learning algorithms is executed. Although the it has high computational cost, the results show that the IDS based on the SVDD-KDE chart produces a high accuracy at 0.917 and AUC at 0.915 with a low false positive rate compared to several algorithms.

FOG-Empowered Augmented-Intelligence-Based Proactive Defensive Mechanism for IoT-Enabled Smart Industries

The recent innovations in the network communication domain have entirely revolutionized the conventional industrial sector by introducing a new era of automatic communication. The Industrial Internet of Things (IIoT) is acknowledged as an exclusive space where IoT seems to divulge expressive manifestations. However, the expanded connectivity, more openness, and the widespread use of low-power communication devices of IIoT makes them vulnerable to malicious attacks, and criminal activities. Moreover, the heterogeneous and prevalent nature of the IIoT devices makes it very difficult to come up with a centralized threat detection mechanism, thus its security remains a major concern. Motivated by this, we propose a fog-empowered Augmented Intelligence (IA)-based defensive mechanism to ensure secure communication in IoT-enabled smart industries. The designed framework incorporates the aggregated potentials of two highly acclaimed Deep Learning (DL) classifiers: Gated Recurrent Unit (GRU) and Bidirectional Long Short-Term Memory (BiLSTM). Where the DL-based scheme detects anomalies in such industrial networks and the FOG-based scenario promotes the routing flexibility and interoperability of the heterogeneous devices of the IoT-enabled smart industrial network. The validity of the proposed mechanism is analytically investigated by conducting a comparison with the benchmark threat detection techniques. The proposed solution is also generically examined parallel to some state-of-the-art approaches. The Cu-GRU-BiLSTM framework achieved up to 99.91% accuracy with a low false alarm rate and outperforms the baseline and recent threat detection approaches. The simulation and comparison results validate the effectiveness of the proposed mechanism and advocate it as a phenomenal choice to ensure efficacious and secure communication in IoT-based smart industries.

Machine Learning Techniques for Detecting Anomalies in IoT Networks

Machine Learning Techniques for Detecting Anomalies in IoT Networks

A comparative evaluation of machine learning algorithms for network anomaly detection

In this era of digital transformation, the importance of network anomaly detection has been amplified to safeguard the security and integrity of various vital applications. These applications include the protection of critical infrastructures, prevention of cyber-attacks, and upkeep of network performance, among others. Our study presents a thorough evaluation of an array of machine learning algorithms for the proficient detection of network anomalies. These algorithms include, but are not limited to, Random Forest, Multi-Layer Perceptron (MLP), and Support Vector Machine (SVM). We illustrate an in-depth comparison between these chosen algorithms, analyzing their performance metrics, strengths, and weaknesses, with a specific focus on their practical applicability and influence on network security methodologies. The investigation into these machine learning algorithms exhibits potential advantages and constraints of employing such methodologies for network anomaly detection. We further shed light on the determinants that affect their performance in diverse scenarios. Based on the exhaustive analysis, we provide suggestive guidelines for choosing the most suitable algorithm depending on specific use cases or requirements. This study thus serves as a comprehensive guide to understanding the role and impact of machine learning in the critical field of network anomaly detection.

A novel deep learning based intrusion detection system for the IoT-Cloud platform with blockchain and data encryption mechanisms

The Internet of Things (IoT) integrated Cloud (IoT-Cloud) has gotten much attention in the past decade. This technology’s rapid growth makes it even more critical. As a result, it has become critical to protect data from attackers to maintain its integrity, confidentiality, protection, privacy, and the procedures required to handle it. Existing methods for detecting network anomalies are typically based on traditional machine learning (ML) models such as linear regression (LR), support vector machine (SVM), and so on. Although these methods can produce some outstanding results, they have low accuracy and rely heavily on manual traffic feature design, which has become obsolete in the age of big data. To overcome such drawbacks in intrusion detection (ID), this paper proposes a new deep learning (DL) model namely Morlet Wavelet Kernel Function included Long Short-Term Memory (MWKF-LSTM), to recognize the intrusions in the IoT-Cloud environment. Initially, to maintain a user’s privacy in the network, the SHA-512 hashing mechanism incorporated a blockchain authentication (SHABA) model is developed that checks the authenticity of every device/user in the network for data uploading in the cloud. After successful authentication, the data is transmitted to the cloud through various gateways. Then the intrusion detection system (IDS) using MWKF-LSTM is implemented to identify the type of intrusions present in the received IoT data. The MWKF-LSTM classifier comes up with the Differential Evaluation based Dragonfly Algorithm (DEDFA) optimal feature selection (FS) model for increasing the performance of the classification. After ID, the non-attacked data is encrypted and stored in the cloud securely utilizing Enhanced Elliptical Curve Cryptography (E2CC) mechanism. Finally, in the data retrieval phase, the user’s authentication is again checked to ensure user privacy and prevent the encrypted data in the cloud from intruders. Simulations and statistical analysis are performed, and the outcomes prove the superior performance of the presented approach over existing models.

Personalized, parcel-guided rTMS for the treatment of major depressive disorder: Safety and proof of concept.

Not all patients with major depressive disorder (MDD) benefit from the US Food and Drug Administration-approved use of repetitive transcranial magnetic stimulation (rTMS) at the dorsolateral prefrontal cortex. We may be undertreating depression with this one-size-fits-all rTMS strategy. We present a retrospective review of targeted and connectome-guided rTMS in 26 patients from Cingulum Health from 2020 to 2023 with MDD or MDD with associated symptoms. rTMS was conducted by identifying multiple cortical targets based on anomalies in individual functional connectivity networks as determined by machine learning connectomic software. Quality of lifeassessed by the EuroQol (EQ-5D) scoreand depression symptomsassessed by the Beck Depression Inventory (BDI) were administered prior to treatment, directly after, and at a follow-up consultation. Of the 26 patients treated with rTMS, 16 (62%) attained remission after treatment. Of the 19 patients who completed follow-up assessments after an average interval of 2.6 months, 11 (58%) responded to treatment and 13 (68%) showed significant remission. Between patients classified with or without treatment-resistant depression, there was no difference in BDI improvement. Additionally, there was significant improvement in quality of life after treatment and during follow-up compared to baseline. This review is retrospective in nature, so there is no control group to assess the placebo effect on patient outcomes. The personalized, connectome-guided approach of rTMS is safe and may be effective for depression. This personalized rTMS treatment allows for co-treatment of multiple disorders, such as the comorbidity of depression and anxiety.

Self-Organizing Computational System for Network Anomaly Exploration using Learning Algorithms

The forum in the nation for reporting information security flaws had 14,871 reports by the end of 2021, a 46.6% increase from 2020. The total of 5,567 high risk vulnerabilities, an increase of nearly 1,400 over the previous year. Evidently, both the total number of vulnerabilities found annually, and the total number of high-risk vulnerabilities are rising. In order for data mining technology to play a wider part in the predictive investigation of network security models, it is advised that its capability have to be improved. This paper combines the concepts of data mining (DM) with machine learning (ML), which introduces similar technologies from DM technology and security establishing collection channel, thereby finally introduces the computer network security maintenance process based on data mining in order to improve the application effect of DM in the predictive analysis of network security models. In this paper, a self-organizing neural network technique that detects denial of service (DOS) in complicated networks quickly, effectively, and precisely is introduced. It also analyses a number of frequently employed computer data mining methods, including association, clustering, classification, neural networks, regression, and web data mining. Finally, it introduces a computer data mining method based on the self-organizing (SO) algorithm. In comparison to conventional techniques, the SO algorithm-based computer data mining technology is also used in defensive detection tests against Dos attacks. A detection average accuracy rate of more than 98.56% and a detection average efficiency gain of more than 20% are demonstrated by experimental data to demonstrate that tests based on the Data Mining connected SO algorithm have superior defensive detection effects than standard algorithms.

Can Strategically Prioritizing Ransomware Protection and Implementing Specific Recommended Actions Potentially Reduce the Effects of a Ransomware Attack on an Organization?

The purpose of this research study was to explore what can be learned by prioritizing for ransomware and data loss protection during organizational strategic planning to mitigate the negative effects of these events while identifying specific actions and recommendations organizations can take to mitigate the effects of ransomware and associated data loss. The problem addressed was that many organizations have become or will become victims of massive data losses caused by ransomware attacks because ransomware security controls and data loss prevention are not prioritized during strategic planning (Breckenridge, 2020). The methodology used was a qualitative metasynthesis which examined six case studies that were chosen as a representative set of organizations that have experienced common effects that are often felt in the aftermath of either a ransomware attack, poor strategic planning for cybersecurity controls, or data loss. The findings show the emergence of several themes including a common lack of strategic planning, the inability to recognize network anomalies leading to malware installations that resulted in data breaches, the lack of or misuse of security controls, and the heavy consequences suffered as a result of not putting effort into preventing the data breaches. Additional examination focused on the need for a proactive approach to data loss prevention which demands the attention of organizational leadership at the highest levels.

Generative Adversarial Network-Based Anomaly Detection and Forecasting with Unlabeled Data for 5G Vertical Applications

With the development of 5G vertical applications, a huge amount of unlabeled network data can be collected, which can be employed for evaluating the user experience and network operation status, such as the identifications and predictions of network anomalies. However, it is challenging to achieve highly accurate evaluation results using the conventional statistical methods due to the limitations of data quality. In this paper, generative adversarial network (GAN)-based anomaly detection and forecasting are studied for 5G vertical applications, which can provide considerable detection and prediction results with unlabeled network data samples. First, the paradigm and deployment of the deep-learning-based anomaly detection and forecasting scheme are designed. Second, the network structure and the training strategy are introduced to fully explore the potential of the GAN model. Finally, the experimental results of our proposed GAN model are provided based on the practical unlabeled network operation data in various 5G vertical scenarios, which show that our proposed scheme can achieve significant performance gains for network anomaly detection and forecasting.

Interactive Learning for Network Anomaly Monitoring and Detection with Human Guidance in the Loop.

With the advancement in big data and cloud computing technology, we have witnessed tremendous developments in applying intelligent techniques in network operation and management. However, learning- and data-based solutions for network operation and maintenance cannot effectively adapt to the dynamic security situation or satisfy administrators' expectations alone. Anomaly detection of time-series monitoring indicators has been a major challenge for network administrative personnel. Monitored indicators in network operations are characterized by multiple instances with high dimensions and fluctuating time-series features and rely on system resource deployment and business environment variations. Hence, there is a growing consensus that conducting anomaly detection with machine intelligence under the operation and maintenance personnel's guidance is more effective than solely using learning and modeling. This paper intends to model the anomaly detection task as a Markov Decision Process and adopts the Double Deep Q-Network algorithm to train an anomaly detection agent, in which the multidimensional temporal convolution network is applied as the principal structure of the Q network and the interactive guidance information from the operation and maintenance personnel is introduced into the procedure to facilitate model convergence. Experimental results on the SMD dataset indicate that the proposed modeling and detection method achieves higher precision and recall rates compared to other learning-based methods. Our method achieves model optimization by using human-computer interactions continuously, which guarantees a faster and more consistent model training procedure and convergence.