Network Traffic Anomalies Research Articles

Multifractal Analysis of Internet Backbone Traffic for Detecting Denial of Service Attacks

This paper proposes to use multifractal analysis to detect backbone network traffic anomalies that indicate network failures or attacks. Multifractal spectrum characteristics are used as security metrics. The effectiveness of the proposed approach is confirmed by experimental results on detecting Denial of Service attacks.

A comprehensive survey on network anomaly detection

Nowadays, there is a huge and growing concern about security in information and communication technology among the scientific community because any attack or anomaly in the network can greatly affect many domains such as national security, private data storage, social welfare, economic issues, and so on. Therefore, the anomaly detection domain is a broad research area, and many different techniques and approaches for this purpose have emerged through the years. In this study, the main objective is to review the most important aspects pertaining to anomaly detection, covering an overview of a background analysis as well as a core study on the most relevant techniques, methods, and systems within the area. Therefore, in order to ease the understanding of this survey’s structure, the anomaly detection domain was reviewed under five dimensions: (1) network traffic anomalies, (2) network data types, (3) intrusion detection systems categories, (4) detection methods and systems, and (5) open issues. The paper concludes with an open issues summary discussing presently unsolved problems, and final remarks.

A Haar Transform-Based Detection Approach to Network Traffic Anomalies in Power Telecommunication Access Networks

This paper proposes a new detection approach to find the abnormal parts in network traffic. Firstly, network traffic is regarded as a discrete time series. Then it is normalized and is carried out the feature component decomposed. Secondly, according to mathematical theory, the feature components in network traffic is effectively refined from the normalized series. The network traffic is divided into feature and residual components. Thirdly, the Haar time-frequency decomposition is carried out for these two components. In this case, a quick anomaly detection algorithm is presented. Simulation results show that our approach is feasible.

Network Traffic Anomaly Detection Using Shallow Packet Inspection and Parallel K-means Data Clustering

Network Traffic Anomaly Detection Using Shallow Packet Inspection and Parallel K-means Data Clustering

NETWORK TRAFFIC ANOMALIES DETECTION BASED ON INFORMATIVE FEATURES

Context. The urgent task for feature informativeness evaluation of a large amount of data has been solved. The object of the study was a network traffic.Objective is to analyze the data informativeness for network traffic anomalies detection in order to reduce the feature space.Method. The approach for feature informativeness evaluation of a large amount of data is proposed to increase the accuracy of the anomaly detection in network traffic. It also substantially increases the computation speed of the classification algorithms. The characteristics of a random forest and Firefly algorithms are considered. In the paper, an algorithm for feature selection based on the integration of these algorithms is proposed. Features are sorted in descending order according to their importance, the least informative ones are not considered. The decision trees, naive Bayes, Bayesian classifier, additive logistic regression and k-nearest neighbors method are considered as classifiers. The quality of the classification results is estimated using six evaluation metrics: true positive rate, false positive rate, precision, recall, Fmeasure and AUC.Results. The experiments have been performed in the Matlab environment (2016a) on the NSL-KDD data set, using the proposed algorithm. The best classification results for the selected features have been obtained using k-nearest neighbors method.Conclusions. The conducted experiments have confirmed the efficiency of the proposed approach and allow recommending it for practical use in feature informativeness evaluation in order to reduce the feature space and increase the computation speed of the classification algorithms. In addition, in order to further study the effectiveness of anomaly detection in network traffic, a real data set will be used.

A Real-Time Detection Approach to Network Traffic Anomalies in Communication Networks

With the advance of new network technologies, new types of applications are quickly arising. Network traffic exponentially is rising. Accordingly, this results in new challenges for network traffic anomaly detections. This paper proposes a new quick detection approach to network traffic anomalies. Firstly, network traffic is regarded as a time series of signals and is constructed into a matrix. Secondly, the principal component decomposition is performed for the matrix. The network traffic is divided into principal and non-principal components. Thirdly, the empirical mode decomposition is carried out for these two components. In this case, a quick anomaly detection algorithm is presented. Simulation results show that our approach is feasible and promising.

Network Traffic Anomaly Detection Based on N-ARMA Model

Network Traffic Anomaly Detection Based on N-ARMA Model

Discovering Fuzzy Exception and Anomalous Rules

Nowadays, searching for specific kind of knowledge that deviates from the usual standards is very useful in several domains such as network traffic anomalies, fraud detection, economic analysis, or medical diagnosis. Fuzzy association rules have been developed as a powerful tool for dealing with imprecision in databases (that may come from the source, i.e., imprecise measures taken by the machine, or from the human understanding of a concept) and offering a comprehensive representation of found knowledge. In this paper, we introduce the notion of fuzzy exception and fuzzy anomalous rule for the recognition of these types of deviations. The deviations are associated with the common patterns, which usually are hidden in data affected by some kind of fuzziness. We present a new approach for mining such rules based on a recently proposed model for representing and evaluating fuzzy rules. Important advantages are to obtain more understandable results and that the mining process can be parallelized. An algorithm following the proposed model is developed, and some experiments are performed in data where some numerical attributes have been fuzzified and also in some real fuzzy transactional datasets for testing the proposed algorithm. From experimentation, we have seen that the proposed fuzzy rules give some insights on the exception and anomaly detection in credit payments.

A hybrid local and distributed sketching design for accurate and scalable heavy key detection in network data streams

Real-time characterization of network traffic anomalies, such as heavy hitters and heavy changers, is critical for the robustness of operational networks, but its accuracy and scalability are challenged by the ever-increasing volume and diversity of network traffic. We address this problem by leveraging parallelization. We propose LD-Sketch, a data structure designed for accurate and scalable traffic anomaly detection using distributed architectures. LD-Sketch combines the classical counter-based and sketch-based techniques, and performs detection in two phases: local detection, which guarantees zero false negatives, and distributed detection, which reduces false positives by aggregating multiple detection results. We derive the error bounds and the space and time complexity for LD-Sketch. We further analyze the impact of ordering of data items on the memory usage and accuracy of LD-Sketch. We compare LD-Sketch with state-of-the-art sketch-based techniques by conducting experiments on traffic traces from a real-life 3G cellular data network. Our results demonstrate the accuracy and scalability of LD-Sketch over prior approaches.

Challenging Entropy-based Anomaly Detection and Diagnosis in Cellular Networks

In this paper we challenge the applicability of entropy-based approaches for detecting and diagnosis network traffic anomalies, and claim that full statistics (i.e., empirical probability distributions) should be applied to improve the change-detection capabilities. We support our claim by detecting and diagnosing large-scale traffic anomalies in a real cellular network, caused by specific OTT (Over The Top) services and smartphone devices. Our results clearly suggest that anomaly detection and diagnosis based on entropy analysis is prone to errors and misses typical characteristics of traffic anomalies, particularly in the studied scenario.

NETWORK TRAFFIC ANOMALIES IDENTIFICATION BASED ON CLASSIFICATION METHODS / TINKLO SRAUTO ANOMALIJŲ IDENTIFIKAVIMAS, TAIKANT KLASIFIKAVIMO METODUS

A problem of network traffic anomalies detection in the computer networks is analyzed. Overview of anomalies detection methods is given then advantages and disadvantages of the different methods are analyzed. Model for the traffic anomalies detection was developed based on IBM SPSS Modeler and is used to analyze SNMP data of the router. Investigation of the traffic anomalies was done using three classification methods and different sets of the learning data. Based on the results of investigation it was determined that C5.1 decision tree method has the largest accuracy and performance and can be successfully used for identification of the network traffic anomalies. Straipsnyje nagrinėjama kompiuterių tinklo srauto anomalijų atpažinimo problema. Apžvelgiami kompiuterių tinklų anomalijų aptikimo metodai bei aptariami jų privalumai ir trūkumai. Naudojant IBM SPSS Modeler programų paketą sudarytas nagrinėjamo tinklo srauto anomalijų atpažinimo modelis, pritaikytas SNMP protokolu pagrįstiems maršruto parinktuvo duomenims apdoroti. Pagal tris klasifikavimo metodus ir skirtingus mokymui skirtus duomenų rinkinius atlikti skaičiavimai tinklo anomalijoms identifikuoti. Palyginant gautus rezultatus nustatyta, kad C5.1 sprendimo medžio algoritmas yra tiksliausias ir sparčiausias, todėl ir tinkamiausias tinklo srauto anomalijoms atpažinti.

Traffic anomaly diagnosis in Internet backbone networks: A survey

Computer networks are becoming increasingly important in supporting business and everyday activities. In particular, the Internet has become part of the critical infrastructure and has a strategic importance in our society and in the digital economy. These developments have led to a highly dynamic network utilization, where traffic fluctuations and seemingly random and anomalous traffic patterns are commonly manifested and hard to diagnose. In order to ensure the protection and resilience of such networks, it is necessary to better analyze and observe network traffic. Thus, anomaly diagnosis aims to discover and characterize critical anomalies affecting the network infrastructure, where the source of these anomalies may be deliberately malicious (e.g. attacks) or unintentional (e.g. failures, misconfigurations or legitimate but abnormal use of the network such as in flash crowds). However, although there is a multitude of algorithms and techniques looking at different elements of the analysis of network traffic anomalies, most research typically focuses on a specific aspect or methodology and there is very little regard for the overall context. This survey aims to present a comprehensive investigation of the current state of the art within the network anomaly diagnosis domain, in particular for Internet backbone networks. We decompose the overall anomaly diagnosis problem spectrum into four main dimensions, namely, processing costs, diagnosis granularity, theoretical methodologies and traffic features. Subsequently the anomaly diagnosis research area is structured further and an overview of the most relevant research is provided by individually reviewing each component of the problem spectrum and proposed solutions with a deeper focus on methodologies and features. Further, we also present and review seminal pieces of work that are considered cornerstones of the anomaly diagnosis research domain.

Detecting anomalies from big network traffic data using an adaptive detection approach

The unprecedented explosion of real-life big data sets have sparked a lot of research interests in data mining in recent years. Many of these big data sets are generated in network environment and are characterized by a dauntingly large size and high dimensionality which pose great challenges for detecting useful knowledge and patterns, such as network traffic anomalies, from them. In this paper, we study the problem of anomaly detection in big network connection data sets and propose an outlier detection technique, called Adaptive Stream Projected Outlier deTector (A-SPOT), to detect anomalies from large data sets using a novel adaptive subspace analysis approach. A case study of A-SPOT is conducted in this paper by deploying it to the 1999 KDD CUP anomaly detection application. Innovative approaches for training data generation, anomaly classification and false positive reduction are proposed in this paper as well to better tailor A-SPOT to deal with the case study. Experimental results demonstrate that A-SPOT is effective and efficient in detecting anomalies from network data sets and outperforms existing detection methods.

NETWORK TRAFFIC ANOMALIES DETECTING USING MAXIMUM ENTROPY METHOD / KOMPIUTERIŲ TINKLO SRAUTO ANOMALIJŲ ATPAŽINIMAS MAKSIMALIOS ENTROPIJOS METODU

The problem of traffic anomalies in computer networks is analyzed. NetFlow packets are used as network traffic data and maximum entropy methods is used for anomalies detection. Method detects network anomalies by comparing the current network traffic against a baseline distribution. Method is adopted according to NetFow data and performace of the method is improved. Prototype of anomalies detection system was developed and experimental investigation carried out. Results of investigation confirmed that method is sensitive to deviations of the network traffic and can be successfully used for network traffic anomalies detection. Straipsnyje nagrinėjama kompiuterių tinklo srauto anomalijų atpažinimo problema. Kompiuterių tinklo srautui stebėti pasirenkama NetFlow technologija, o anomalijos aptinkamos maksimalios entropijos metodu. Metodas pritaikytas NetFlow pateikiamiems duomenims apdoroti. Sukurta programinė priemonė ir atliktas eksperimentinis metodo tinkamumo tyrimas analizuojant „Cisco“ maršrutizatoriaus srauto duomenis. Metodas patobulintas siekiant pagreitinti skaičiavimus, tačiau neprarandant tikslumo. Nustatyta, kad metodas yra jautrus įvairaus tipo tinklo srauto nuokrypiams ir gali būti sėkmingai taikomas tinklo srauto anomalijoms aptikti.

Uncovering network traffic anomalies based on their sparse distributions

Characterizing network traffic with higher-dimensional features results in increased complexity of most detectors and classifiers for identifying traffic anomalies. Several key observations from existing studies confirm that network anomalies are typically distributed in a sparse way, with each anomaly essentially characterized by its lower-dimensional features. Based on this important finding, we exploit sparsity in designing a novel detection method for anomalies that ignores redundancies that are dynamically filtered from the feature sets and accurately classifies anomalies. Comparison of our method with three well known techniques shows a 10% improvement in accuracy with an O (n) complexity of the classifier.

Spatiotemporal model for Internet traffic anomalies

Models for Internet traffic anomalies greatly benefit a range of applications including robust network design, network provisioning and performance studies. A novel approach to analyse and model network traffic anomalies is presented. The proposed approach individually characterises different aspects of anomalies, such as origin, termination, propagation and changes in duration and volume, with common random processes. These characteristics are then integrated into a single model that successfully captures the overall anomaly behaviours. Characterisation of each anomaly property requires only a few parameters, leading to a concise set of parameters for the entire model. Although the model is calibrated with local measurements made at nodes, it successfully represents the global behaviours of anomalies over the network. The proposed model is applicable both at nodal level and at subnet level. This enables hierarchically analysing large and sophisticated networks. Anomalies are analysed using a multi-scale analysis framework based on which, a real-time monitoring system that efficiently communicate ongoing anomaly information across the network is developed. The system is also used for learning regional model parameters distributively. Internet2 traffic data is analysed using the framework, and the corresponding model parameters are derived. These results provide insight on the nature of anomalies in networks.

A Hybrid Technique Using PCA and Wavelets in Network Traffic Anomaly Detection

Research into network anomaly detection has become crucial as a result of a significant increase in the number of computer attacks. Many approaches in network anomaly detection have been reported in the literature, but data or solutions typically are not freely available. Recently, a labeled network traffic flow dataset, Kyoto2006+, has been created and is publicly available. Most existing approaches using Kyoto2006+ for network anomaly detection apply various clustering techniques. This paper leverages existing well known statistical analysis and spectral analysis techniques for network anomaly detection. The first popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. The other well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA–Haar Wavelet Analysis. The hybrid approach first applies PCA to describe the data and then Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA–Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. The authors consider a number of parameters and present experimental results to demonstrate the effectiveness of the hybrid approach as compared to the two algorithms individually.

The Effectively Method of Detecting Network Traffic Anomaly

A kind of detection and location method for the network traffic anomaly focusing the wavelet analysis was proposed in this paper on the signal feature and self-similarity of the network traffic in the enterprise. The method can effectively detect the network traffic anomaly and locate the anomaly point through the simulation and verification of the data collected in DipSIF platform.

A New Method of Detecting Network Traffic Anomalies

As Internet communications and applications become more and more complex, accurately describing network traffic information and rapidly monitoring network traffic anomalies have become increasingly challenging tasks. In this paper, we present a framework and method for monitoring network traffic through measuring the dynamic changes of host communities. An unweighted and undirected host interaction network (HIN) is established through extracting the social-behavioral characteristics of network traffic. Based on social-behavior similarity in HIN, host community is defined, and then five features are proposed to capture host community changes. Finally, this method is evaluated through two real-world network traffic, and the experimental results show that the method presented in this paper can effectively capture the dynamic changes of host communities to monitor network traffic.

Anomaly secure detection methods by analyzing dynamic characteristics of the network traffic in cloud communications

Cloud computing represents a new paradigm where computing resources are offered as services in the world via communication Internet. As many new types of attacks are arising at a high frequency, the cloud computing services are exposed to an increasing amount of security threats. To reduce security risks, two approaches of the network traffic anomaly detection in cloud communications have been presented, which analyze dynamic characteristics of the network traffic based on the synergetic neural networks and the catastrophe theory. In the former approach, a synergetic dynamic equation with a group of the order parameters is used to describe the complex behaviors of the network traffic system in cloud communications. When this equation is evolved, only the order parameter determined by the primary factors can converge to 1. Then, the anomaly can be detected. In the latter approach, a catastrophe potential function is introduced to describe the catastrophe dynamic process of the network traffic in cloud communications. When anomalies occur, the state of the network traffic will deviate from the normal one. To assess the deviation, an index named as catastrophe distance is defined. The network traffic anomaly can be detected by the value of this index. We evaluate the performance of these two approaches using the standard Defense Advanced Research Projects Agency data sets. Experimental results show that our approaches can effectively detect the network traffic anomaly and achieve the high detection probability and the low false alarms rate.