IP Flow Research Articles

NetTiSA: Extended IP flow with time-series features for universal bandwidth-constrained high-speed network traffic classification

Network traffic monitoring based on IP Flows is a standard monitoring approach that can be deployed to various network infrastructures, even the large ISP networks connecting millions of people. Since flow records traditionally contain only limited information (addresses, transport ports, and amount of exchanged data), they are also commonly extended by additional features that enable network traffic analysis with high accuracy. These flow extensions are, however, often too large or hard to compute, which then allows only offline analysis or limits their deployment only to smaller-sized networks. This paper proposes a novel extended IP flow called NetTiSA (Network Time Series Analysed) flow, based on analysing the time series of packet sizes. By thoroughly testing 25 different network traffic classification tasks, we show the broad applicability and high usability of NetTiSA flow. For practical deployment, we also consider the sizes of flows extended by NetTiSA features and evaluate the performance impacts of their computation in the flow exporter. The novel features proved to be computationally inexpensive and showed excellent discriminatory performance. The trained machine learning classifiers with proposed features mostly outperformed the state-of-the-art methods. NetTiSA finally bridges the gap and brings universal, small-sized, and computationally inexpensive features for traffic classification that can be scaled up to extensive monitoring infrastructures, bringing the machine learning traffic classification even to 100Gbps backbone lines.

Markov-Modulated On–Off Processes in IP Traffic Modeling

This paper deals with the modeling of real IP flows using Markov-modulated On–Off processes. In the first section of the paper, we summarize the knowledge found so far about the Markov modulated On–Off regular process model, which has already been published in our previous papers. For the sake of completeness, we also summarize the well-known facts regarding the Bernoulli process. In the second section, we deal with the continuation of modeling using the Markov-modulated On–Off Bernoulli process. Our own derivation of the hitherto-unknown probability distribution of time spaces (tail distribution) is completely new. For its derivation, we used the tail distribution generating function, and then, using its derivation, we calculated the hitherto-unknown moments of the distribution (mean, variation, and third initial moment). This knowledge will allow us to create a new numerical procedure for estimating MMBP parameters from measured IP traffic. Finally, we present a formula for the sizing of network resources for a given flow using effective bandwidth with respect to QoS based on a given level of IP traffic.

Combining Stochastic and Deterministic Modeling of IPFIX Records to Infer Connected IoT Devices in Residential ISP Networks

Residential Internet service providers (ISPs) today have limited device-level visibility into subscriber houses, primarily due to the network address translation (NAT) technology. The continuous growth of “unmanaged” consumer Internet of Things (IoT) devices combined with the rise of work-from-home makes home networks attractive targets to sophisticated cyber attackers. Volumetric attacks sourced from a distributed set of vulnerable IoT devices can impact ISPs by deteriorating the performance of their network, or even making them liable for being a carrier of malicious traffic. This article explains how ISPs can employ IP Flow Information eXport (IPFIX), a flow-level telemetry protocol available on their network, to infer connected IoT devices and ensure their cyber health without making changes to home networks. Our contributions are threefold: 1) we analyze more than nine million IPFIX records of 26 IoT devices collected from a residential testbed over three months and identify 28 flow features pertinent to their network activity that characterize the network behavior of IoT devices—we release our IPFIX records as open data to the public; 2) we train a multiclass classifier on stochastic attributes of IPFIX flows to infer the presence of certain IoT device types in a home network with an average accuracy of 96%. On top of the machine learning (ML) model, we develop a trust metric to track network activity of detected devices over time; and 3) finally, we develop deterministic models (DTs) of specific and shared cloud services consumed by IoTs, yielding an average accuracy of 92%. We show a combination of stochastic and DTs mitigates false positives in 75% of incidents at the expense of an average 7% reduction in true positives.

Network Transmission Flags Data Affinity-based Classification by K-Nearest Neighbor

Abstract—This research is concerned with the data generated during a network transmission session to understand how to extract value from the data generated and be able to conduct tasks. Instead of comparing all of the transmission flags for a transmission session at the same time to conduct any analysis, this paper conceptualized the influence of each transmission flag on network-aware applications by comparing the flags one by one on their impact to the application during the transmission session, rather than comparing all of the transmission flags at the same time. The K-nearest neighbor (KNN) type classification was used becauseit is a simple distance-based learning algorithm that remembers earlier training samples and is suitable for taking various flags withtheir effect on application protocols by comparing each new sample with the K-nearest points to make a decision. We used transmission session datasets received from Kaggle for IP flow with 87 features and 3.577.296 instances. We picked 13 features from the datasets and ran them through KNN. RapidMiner was used for the study, and the results of the experiments revealed that the KNN-based model was not only significantly more accurate in categorizing data, but it was also significantly more efficient due to the decreased processing costs.

Adversarial Deep Learning approach detection and defense against DDoS attacks in SDN environments

Over the last few years, Software Defined Networking (SDN) paradigm has become an emerging architecture to design future networks and to meet new application demands. SDN provides resources for improving network control and management by separating control and data plane, and the logical control is centralized in a controller. However, the centralized control logic can be an ideal target for malicious attacks, mainly Distributed Denial of Service (DDoS) attacks. Recently, Deep Learning has become a powerful technique applied in cybersecurity, and many Network Intrusion Detection (NIDS) have been proposed in recent researches. Some studies have indicated that deep neural networks are sensitive in detecting adversarial attacks. Adversarial attacks are instances with certain perturbations that cause deep neural networks to misclassify. In this paper, we proposed a detection and defense system based on Adversarial training in SDN , which uses Generative Adversarial Network (GAN) framework for detecting DDoS attacks and applies adversarial training to make the system less sensitive to adversarial attacks. The proposed system includes well-defined modules that enable continuous traffic monitoring using IP flow analysis, enabling the anomaly detection system to act in near-real-time. We conducted the experiments on two distinct scenarios, with emulated data and the public dataset CICDDoS 2019. Experimental results demonstrated that the system efficiently detected up-to-date common types of DDoS attacks compared to other approaches.

Enhanced Bit Repair IP Fast Reroute Mechanism for Rapid Network Recovery

The massive development of virtualized infrastructures, Internet of Things (IoT), and Wireless Sensor Network (WSN) in recent years has led to an increase in quality requirements for the management and reliability of underlay communication networks. Existing converged networks must therefore guarantee specific quantitative and qualitative parameters of different network communication services to meet customer requirements. However, the quality of the services operated is very negatively affected by an unpredictable failure of a communication link or a network node. In such situations, communication is typically interrupted for a period that is difficult to predict, and which can lead to significant financial losses and other negative effects. Internet Protocol Fast Reroute (IP FRR) technology was developed for these reasons. The paper presents the proposal of the new Enhanced Bit Repair (EB-REP) IP FRR mechanism, which offers significant improvements over its predecessor, the B-REP mechanism. The B-REP offers protection against a single failure and only for selected critical IP flows. The EB-REP provides advanced protection against multiple failures in a protected network domain and the protection can be provided for all network flows. The EB-REP calculates alternative paths in advance based on link metrics, but also allows the construction of alternative paths independently of them. The construction of alternative FRR paths uses a standardized tunneling approach via a unique field Bit-String. Thanks to these features, EB-REP is an advanced contribution to solving IP FRR-related problems, which enables the use of EB-REP in many network deployments, but especially in network solutions that require reliable data transmission.

Auxiliary-Graph-Based Energy-Efficient Traffic Grooming in IP-Over-Fixed/Flex-Grid Optical Networks

Due to the development of broadband access technologies including FTTx and 5G, the traffic in metro and core networks is forecasted to be more dynamic. To support this dynamicity, flex-grid elastic optical networks (EONs) have started to become prolific over the past few years. During the migration from the legacy fixed-grid wavelength division multiplexing optical networks to flex-grid EONs, a transition technology called the fixed/flex-grid optical network has become popular. In such fixed/flex-grid networks, the energy consumption caused by serious capacity mismatch between variable IP traffic and the different optical channel capacities is significantly high to be ignored. In fact, energy-efficient traffic grooming (i.e., to aggregate multiple fine-grained IP flows and map them onto a lightpath) is a challenge in IP-over-fixed/flex-grid optical networks because co-existing fixed-grid and flex-grid nodes have different traffic grooming ability and energy consumption models. To overcome the challenge, we propose an auxiliary-graph-based energy-efficient traffic-grooming (EETG) algorithm in IP-over-fixed/flex-grid optical networks for the first time. The construction of an auxiliary graph (AG) model for each connection request is one of the core steps. The well-designed weight-assignment scheme of the AG establishes the relationships between traffic grooming operations and specific energy consumption models. The final traffic grooming operations are undertaken based on the calculated weight-minimized path. The simulation results show that the EETG algorithm achieves 1) the least energy consumption compared with the state-of-the-art traffic grooming algorithms under the non-blocking scenario; and 2) achieves the tradeoff between energy consumption and blocking probability under the blocking scenario.

A GRU deep learning system against attacks in software defined networks

The management of modern network environments is becoming more and more complex due to new requirements of devices' heterogeneity regarding the popularization of the Internet of Things (IoT), as well as the dynamic traffic required by next-generation applications and services. To address this problem, Software-defined Networking (SDN) emerges as a management paradigm able to handle these problems through a centralized high-level network approach. However, this centralized characteristic also creates a critical failure spot since the central controller may be targeted by malicious users aiming to impair the network operation. This paper proposes an SDN defense system based on the analysis of single IP flow records, which uses the Gated Recurrent Units (GRU) deep learning method to detect DDoS and intrusion attacks. This direct flow inspection enables faster mitigation responses, minimizing the attack's impact over the SDN. The proposed model is tested against several different machine learning approaches over two public datasets, the CICDDoS 2019 and the CICIDS 2018. Furthermore, a lightweight mitigation approach is presented and evaluated through performance tests regarding each detection method. Finally, a feasibility test is performed regarding the throughput of flows per second that each detection method can analyze. This test is accomplished through the use of real IP Flow data collected at a large-scale network. The results point out promising detection rates and an elevated amount of analyzed flows per second, which makes GRU a feasible approach for the proposed system.

D-Sketch: A Differentiated Sketch Strategy for Double-Stack Networks

Representing traffic with sketch data structures to provide flow statistics is a fundamental strategy in network measurement. In the current double-stack circumstance, we observe that IPv4 and IPv6 flows are inequivalent in terms of both flow cardinality and flow size in Internet. The existing sketches, however, represent IPv4 and IPv6 flows with no differentiation. As a consequence, the IPv4 flows, which can be massive and large, while the IPv6 flows, which are usually handful and small. Such an unbalance or asymmetry feature may lead to unnecessary hash collision errors, especially for IPv6 flows. To this end, in this letter, we present D-Sketch, a new sketch optimization strategy to represent IP flows. At its core, D-Sketch differentiates IPv4 flows from IPv6 ones, thereafter represents them with isolated sketches whose capacities are proportional to the corresponding flow cardinality. Given the same space overhead, trace-driven evaluations further commit that D-Sketch decreases the ARE of per-flow measurement up to 52.5%, with an average of 24.0%, compared with the existing one-sketch and large-small strategies.

How to Mislead AI-Assisted Network Automation in SD-IPoEONs: A Comparison Study of DRL- and GAN-Based Approaches

Recently, the combination of artificial intelligence (AI) and software-defined networking (SDN) has attracted intensive research interests because it realizes and promotes AI-assisted network automation (AIaNA). Despite the initial successes of AIaNA, its vulnerabilities, i.e., the downside of the reduction of human involvement achieved by it, have not been carefully explored. In this work, we use software-defined IP over elastic optical networks (SD-IPoEONs) as the background, and study how to mislead the AIaNA system in them. Specifically, we target our attack on the deep neural network (DNN) based traffic predictor in the AIaNA system, and design an adversarial module (ADVM) that can craft and inject adversarial traffic samples adaptively to disturb its operation. We consider two approaches to design the ADVM, i.e., the deep reinforcement learning (DRL) based on deep deterministic policy gradient (DDPG), and the generative adversarial network (GAN) model. Our proposed ADVM can monitor and interact with a dynamic SD-IPoEON to train itself on-the-fly. This enables it to generate and inject adversarial samples in the most disturbing and hard-to-detect way and to severely affect the AIaNA's performance on multilayer service provisioning. Specifically, IP flows will be served incorrectly to result in unnecessary congestions/under-utilizations on lightpaths, and erroneous network reconfigurations will be invoked frequently. Simulation results confirm the effectiveness of our ADVM designs, and show that the GAN-based ADVM achieves better attack effects with smaller perturbation strength.

3,6-dichloro-1,2,4,5-Tetrazine Assayed at High Doses in the Metastatic Breast Cancer Cell Line MDA-MB-231 Reduces Cell Numbers and Induces Apoptosis

Background: The lack of treatment options for patients with chemotherapy-resistant cancers is pushing the field toward the development of new therapies. 1,2,4,5-tetrazine derivatives are a class of heterocyclic compounds that exhibit a broad spectrum of antitumor activities. Objective: The purpose of this study was to assess the biological activity of four s-tetrazine derivatives by substitution of two chloride atom of 3,6-dichloro-1,2,4,5-tetrazine with long hydrophobic side chains. Methods: We analyzed the anti-proliferative effects of four s-tetrazine derivatives with MTT assay and their pro-apoptotic effect with AV/ IP flow cytometry analysis and Hoechst 33342 staining. Results: We demonstrated that 3,6-dichloro-1,2,4,5-tetrazine (compound (1)) has a cytotoxic effect and induces apoptosis. Conclusion: 3,6-dichloro-1,2,4,5-tetrazine presents a new cytotoxic drug against metastatic breast cancer cell line MDA-MB-231 in vitro.

Flow based monitoring of ICS communication in the smart grid

A smart grid network is a part of critical infrastructure, and its interruption or blackout may cause fatal consequences on energy production, distribution, and eventually lives of people. Smart grid networks can be a target of cyber attacks coming from the outside or the inside of the network. Traditional smart grid protection includes firewalls and IDS/IPS devices that are usually deployed on edges of the network where they inspect incoming and outgoing traffic. This approach is adequate to cope with external threats. In case of internal threats caused by, for instance, the malware infecting the control station, it is not easy to detect malicious activity commonly masked as legitimate communication at the network edge. For the successful identification of cyber security attacks, two essential elements are necessary.The first is the visibility of the Industrial Control System (ICS) communication, which enables a smart grid operator to see real-time transmissions in the network. The second important part is an anomaly detection system that analyzes monitoring data and identifies possible cyber security attacks. This paper presents a novel system for monitoring ICS/SCADA protocols based on IP flows extended with application layer data obtained from ICS packet headers. The monitoring system provides an in-depth insight into ICS communication. By applying statistical-based methods or creating communication profiles using probabilistic automata, common security attacks, as well as unknown threats, can be identified. The proposed approach is demonstrated on IEC 60870-5-104 communication.

Why are My Flows Different? A Tutorial on Flow Exporters

Network flows build the basis of modern network data analysis by aggregating properties of network packets with common characteristics. A consistent and unambiguous definition of the network flow concept is an indispensable prerequisite and starting point for reproducible network research. However, in today’s practice, the flow output of distinct flow exporters, which is software to generate flows from observed network packets, varies substantially on identical network packet stream input. In this paper we present an in-depth comparison of different flow exporters and show how their outputs differ significantly. We argue that this substantially impairs reproducibility for traffic analysis research. We first present the detailed flow definition of the IP Flow Information eXport (IPFIX) standard including explanations and examples, analyze design and implementation of existing flow exporters, and explore the reasons why many projects and publications chose to implement their own flow exporters. Based on this analysis we highlight the main challenges in the flow exporting process and present a detailed tutorial on how to design and implement a flow exporter such that it yields consistent, reproducible output. Based on the tutorial’s theoretical analysis and lessons learned we present design and main concepts of a versatile, flexible, and open source flow exporting solution called go-flows that generates deterministic, reproducible network flows. Finally, we present a flow-by-flow comparison of the analyzed flow exporters’ output, explore the differences in terms of their generated flows, compare flow exporter performance, and conclude with guidelines on parameters that play a crucial role in improving the reproducibility of exported flows.

IP backbone traffic behavior characteristic spectrum composing and role mining

The discovery and description of the IP traffic behavior is of great significance for both network operation management and network security monitoring. Researches demonstrate that there are some similarities of the traffic behavior among different IPs, hence, they can be clustered based on the behavior similarity. These similar traffic behaviors can be depicted by a specific behavior pattern called IP address role in our work. Towards this end, a unidirectional IP flow record is used to represent an independent IP activity. The traffic behavior metrics are defined in four dimensions including the duration time, the peer address, the application types and the number of packets and bytes contained in the flow, which corresponds to temporal dimension, spatial dimension, category dimension and intensity dimension, respectively. Nine single-attribute and thirty-nine dual-attribute metrics are extracted from four dimensions to compose the IP address traffic characteristic spectrum, which is used to profile the behavior of all IPs in the observed network and provide the data for the behavior description of each class of IP. These classes are established by a characteristic spectrum matched IP address role mining algorithm designed in this paper. NetFlow data collected from some border routers of China Education Research Network backbone (CERNET) is used to verify the method. Experimental results demonstrate that our approach can be applied to anomaly behavior detection and mainstream behavioral habits analysis.

Throughput and Delay Analysis of LWA With Bursty Traffic and Randomized Flow Splitting

We investigate the effect of bursty traffic in a long term evolution (LTE) and Wi-Fi aggregation (LWA)-enabled network. The LTE base station routes packets of the same IP flow through the LTE and Wi-Fi links independently. We motivate the use of superposition coding at the LWA-mode Wi-Fi access point (AP) so that it can serve LWA users and Wi-Fi users simultaneously. A random access protocol is applied in such system, which allows the native-mode AP to access the channel with probabilities that depend on the queue size of the LWA-mode AP to avoid impeding the performance of the LWA-enabled network. We analyze the throughput of the native Wi-Fi network and the delay experienced by the LWA users, accounting for the native-mode AP access probability, the traffic flow splitting between LTE and Wi-Fi, and the operating mode of the LWA user with both LTE and Wi-Fi interfaces. Our results show some fundamental tradeoffs in the throughput and delay behavior of LWA-enabled networks, which provide meaningful insight into the operation of such aggregated systems.

Visualize Your IP-Over-Optical Network in Realtime: A P4-Based Flexible Multilayer In-Band Network Telemetry (ML-INT) System

Nowadays, with the fast development of backbone networks, performance monitoring and troubleshooting on IP-over-optical networks have become increasingly important. However, a real-time monitoring scheme, which is programmable to reveal the end-to-end multilayer information of an arbitrary flow in an IP-over-optical network, is still absent. Also, for such a monitoring scheme, how to balance the tradeoff between the accuracy and the overhead has not been studied yet. To address these challenges, we design a P4-based flexible multilayer in-band network telemetry (ML-INT) system to visualize an IP-over-optical network in real time. Specifically, the flexible ML-INT scheme only selects a small portion of packets in an IP flow to encode INT headers, while each INT header only contains a part of the statistics of all the electrical/optical network elements (NEs) on the flow’s routing path. We design the packet processing pipelines for the scheme, program P4-based hardware programmable data-plane (PDP) switches to implement the pipelines, develop the optical performance monitors that can cooperate with the PDP switches to facilitate ML-INT, and implement a high-performance data analyzer that can extract, parse, and analyze the INT data carried by the high-speed IP flows [i.e., with an arrival rate up to 2 million packets per second (Mp/s)]. The whole flexible ML-INT system is experimentally demonstrated in a small-scale but real IP-over-optical network testbed. The experimental results verify that our proposal only introduces very small overhead and can make the IP-over-optical network more visible in real time for performance monitoring and troubleshooting.

Proportional and Preemption-Enabled Traffic Offloading for IP Flow Mobility: Algorithms and Performance Evaluation

IP flow mobility (IFOM) enables a user equipment to offload data traffic at the IP flow level. Although the procedure of IFOM-based flow offloading has been specified by 3GPP, how many IP flows should be offloaded and when offloading should be performed are not defined. Consequently, IP flows may be routed to a target access network that has a strong signal strength but with backhaul congestion or insufficient access capability. In this paper, we propose two algorithms, referred to as proportional offloading (PO) and proportional and preemption-enabled offloading (PPO), respectively, for IP flow offloading in hybrid cellular and wireless local area networks. The PO algorithm decides an optimal proportion of IP flows, which could be offloaded by considering available resources at the target access network. In the PPO algorithm, both service continuity and network utilization are taken into consideration. Furthermore, a detailed analytical model is developed in order to evaluate the behavior of the proposed algorithms. The analytical model is validated through extensive simulations. The results show that by dynamically adjusting the percentage of traffic flows to be offloaded, PO can reduce blocking probability and increase resource utilization. PPO further improves the performance at the cost of slightly higher offloading overhead.

Experimental Evaluation of a SDN-DMM Architecture

Mobility management has become a great challenge due to the exponential growth in the number of devices that can connect to home or visited networks, and the need for providing seamless mobility in future generation networks. SDN-DMM (Software Defined Network Architecture for Distributed Mobility Management) architecture has been proposed [11], allowing to separate control and data planes, for the distributed mobility management through bidirectional IP flows. This article reports on aspects related to the implementation of SDN-DMM, conducted with metrics as packet loss, throughput and handover latency, considered in a comparison involving traditional routing and SDN-DMM. The results show the SDN approach not only provides the intrinsic benefits of SDN in comparison with traditional architectures, but also deals with the distributed mode of mobility management in heterogeneous access networks in a simplified and efficient way.

Fast Defense System Against Attacks in Software Defined Networks

With the ever-growing data traffic in computer networks nowadays, the management of large-scale networks is a challenge for guaranteeing the quality of the provided services. This is due to the increasingly usage of connected applications, such as Internet of Things and cloud computing environments. Software-defined networking (SDN) is a new paradigm that aims to make this management process easier by centralizing the configuration of all network devices into a single programmable central controller. However, as any centralized service, this architecture is susceptible to security vulnerabilities, such as distributed denial of service (DDoS) and port scan attacks. Thus, security methods are necessary to guarantee the normal operation of SDN’s central controller. Furthermore, networks are transporting an increasingly amount of information day by day, which could mean data loss in case of long network unavailability. For this reason, security mechanisms must operate online, with fast-responding countermeasures to mitigate the impact of the detected attacks over the SDN. In this paper, we present a fast SDN defense system against DDoS and port scan attacks, which runs directly into the central controller and uses a game theoretical approach for attack mitigation. For the detection, we compare three different approaches, particle swarm optimization, multi-layer perceptron neural network, and discrete wavelet transform. We test our approach over IP flow data generated over Mininet network emulator, along with floodlight controller, and the presented defense system achieved good outcomes for both detection and mitigation processes.

Web Phishing Detection Using a Deep Learning Framework

Web service is one of the key communications software services for the Internet. Web phishing is one of many security threats to web services on the Internet. Web phishing aims to steal private information, such as usernames, passwords, and credit card details, by way of impersonating a legitimate entity. It will lead to information disclosure and property damage. This paper mainly focuses on applying a deep learning framework to detect phishing websites. This paper first designs two types of features for web phishing: original features and interaction features. A detection model based on Deep Belief Networks (DBN) is then presented. The test using real IP flows from ISP (Internet Service Provider) shows that the detecting model based on DBN can achieve an approximately 90% true positive rate and 0.6% false positive rate.