Network Management Applications Research Articles

Deep Generative Model and Its Applications in Efficient Wireless Network Management: A Tutorial and Case Study

Deep Generative Model and Its Applications in Efficient Wireless Network Management: A Tutorial and Case Study

ActiveGuardian: An accurate and efficient algorithm for identifying active elephant flows in network traffic

Active elephant flows, which indicate the real-time data transmission status, are of primary interest in network management and various applications. However, existing network measurement efforts mainly focus on finding elephant flows, and limited works on identifying active elephant flows suffer from low accuracy and heavy overheads. To address this issue, this paper proposes ActiveGuardian to identify active elephant flows with high accuracy, low memory, and high throughput. The key idea is to intelligently separate potential elephant flows from mice flows, and guard and report the information of active elephant flows in every time window. To obtain high accuracy, we devise a filtering module that adaptively filters unnecessary flows with low arrival rates, by applying an Adaptive Counter Update strategy. To achieve high memory utilization, we design a Leapfrog Hashing algorithm for the guarding module to effectively solve hash collisions. Lastly, we perform theoretical derivation on the false positive and the error bound of ActiveGuardian, and experimental evaluations on its performance with real network traffic traces. The experimental results show that ActiveGuardian achieves higher accuracy (99.65%) with identical memory sizes, and higher throughput (26.53Mps) than the state-of-the-art solutions.

A Scalable Monitoring System for Software Defined Networks

Monitoring functionality is an essential element of any network system. Traditional monitoring solutions are mostly used for manual and infrequent network management tasks. Software-defined networks (SDN) have emerged with enabled automatic and frequent network reconfigurations. In this paper, a scalable monitoring system for SDN is introduced. The proposed system monitors small, medium, and large-scale SDN. Multiple instances of the proposed monitoring system can run in parallel for monitoring many SDN slices. The introduced monitoring system receives requests from network management applications, collects considerable amounts of measurement data, processes them, and returns the resulting knowledge to the network management applications. The proposed monitoring system slices the network (switches and links) into multiple slices. The introduced monitoring system concurrently monitors applications for various tenants, with each tenant's application running on a dedicated network slice. Each slice is monitored by a separate copy of the proposed monitoring system. These copies operate in parallel and are synchronized. The scalability of the monitoring system is achieved by enhancing the performance of SDN. In this context, scalability is addressed by increasing the number of tenant applications and expanding the size of the physical network without compromising SDN performance.

Toward Low-Latency and Accurate State Synchronization for Programmable Networks

Programmable switches empower stateful packet processing, in which incoming packets continuously update states in the data plane, while applications in the control plane read and write states. However, since the data plane and control plane are separated, a consistent view of states in both planes is required for stateful packet processing. Existing approaches suffer from either high latency or low accuracy. In this paper, we propose, a framework that offers approximate state synchronization with low latency and high accuracy. To achieve low latency, directly transfers states between switch ASICs and the control plane by bypassing switch operating systems. To achieve high accuracy, utilizes the resources in the switch ASIC to realize rate control in state synchronization, such that it avoids potential state loss. It also bounds the divergence between the states in the data plane and that in the control plane under limited link capacity. We prototype on Barefoot Tofino switches. The experimental results indicate that compared to existing approaches, achieves order-of-magnitude latency reduction while maintaining high accuracy of state synchronization. Also, our experiments demonstrate that provides significant latency benefits to existing network management applications and well preserves high application-level accuracy.

A novel data streaming method for detecting abnormal flows in distributed monitoring systems

AbstractThis paper concentrates on the issue of detecting abnormal flows in distributed monitoring systems, which has many network management applications such as anomaly detection and traffic engineering. Collecting massive network traffic in real‐time remains a large challenge due to the limited system resource. Most existing approaches perform abnormal flow detection at one measurement point, while they cause large computation and memory overhead for recovering abnormal flows. In this paper, we propose a novel data streaming method that supports accurate abnormal flow detection with a low memory requirement. The key idea of our method is that each monitor compresses flow information to summary data structure, sends the generated data structure to the controller; then the controller aggregates the received data structures, recovers candidates of abnormal flows and estimates their size and change to find abnormal flows on the basis of the aggregated data structure. The experimental results based on real network traffic show that the proposed approach can detect up to 97% of abnormal flows with low memory and update requirements in comparison with related approaches.

A RYU-SDN Controller-Based VM Migration Scheme Using SD-EAW Ranking Methods for Identifying Active Jobs in the 5G Cloud Framework

The presented scheme focuses on active jobs live migration among VMs in 5G cloud framework depending on the software defined networks (SDN) to improve QoS in cloud framework. In this approach, RYU SDN controller is employed, which provides software components that allows software developers to extend network management and control applications for utilizing the features of SDN controller. It currently supports variety of southbound protocols such as OpenFlow, OF-Config, NETCONF, etc., whereas the proposed system uses Mininet prototype network. The destination server selection in the data centre is based on the server distinction based equivalent active weights (SD-EAW) ranking methods. The weight computation necessitate was to recognize non-active and active jobs. A presented SD-EAW scheme utilizes Pareto distribution for the recognition of active and inactive jobs in both continuous and discrete intervals of time. The presented SD-EAW algorithm functions well over all traditional approaches and in turn offers an optimum solution through minimizing the cloud environment's make span.

SECURING CENTRALIZED SDN CONTROL WITH DISTRIBUTED BLOCKCHAIN TECHNOLOGY

Software Defined Networks (SDN) advocates segregation of network control logic, forwarding functions and management applications into different planes to achieve network programmability, automated and dynamic flow control in next generation networks. It promotes deployment of novel and augmented network management functions to have flexible, robust, scalable and cost-effective network deployments. All these features introduce new research challenges and require secure communication protocols among the segregated network planes. This manuscript focuses on the security issue of southbound interface which operates between the SDN control and data plane. We have highlighted the security threats associated with an unprotected southbound interface and the issues related with the existing TLS based security solution. A lightweight blockchain based decentralized security solution is proposed for southbound interface to secure the resources of logically centralized SDN controllers and distributed forwarding devices from opponents. The proposed mechanism can operate in multi-domain SDN deployment and can be used with wide range of network controllers and data plane devices. In addition to it, the proposed security solution is analyzed in terms of security features, communication and reauthentication overhead.

Resource Orchestration at the Edge: Intelligent Management of mmWave RAN and Gaming Application QoE Enhancement

Millimeter wave (mmWave) is a crucial component in 5G and beyond 5G communications. However, the dense deployment of mmWave transceivers would impose a heavy burden on the management of the radio access network (RAN). This challenge increases the need for leveraging intelligent network management techniques. Thanks to edge computing, machine learning (ML) based network management algorithms and other delay-sensitive user applications can operate at the network edge. But, due to the limited resources on edge servers, developing an orchestration scheme for intelligent network management and user applications is necessary. In this paper, we provide an edge-centric resource management framework for intelligent RAN management and applications with the awareness of the users’ quality of experiences (QoE). Specifically, we consider the scenario of a mmWave communication system equipped with an ML-based mmWave beam tracking algorithm. The users under this system request mobile edge gaming services. We formulate a game QoE aware orchestration problem as a non-linear integer programming and prove its NP-hardness. To reduce the complexity, we decompose the original problem into two subproblems, the service placement problem for mobile edge gaming and the configuration selection and placement problem for mmWave beam tracking. Then, we solve the two subproblems consecutively with heuristic approaches. Simulation results demonstrate the effectiveness of the proposed orchestration scheme.

FEAT: A Federated Approach for Privacy-Preserving Network Traffic Classification in Heterogeneous Environments

Network traffic classification is the foundation for many network security and network management applications. Recently, to preserve the privacy of the data which are generated in the mobile ends, federated learning (FL)-based classification methods are being proposed. Unfortunately, the performance of FL-based methods can seriously degrade when the client data have skewness. This is particularly true for mobile network traffic classification where the environments in the mobile ends are highly heterogeneous. In this article, we first conduct a measurement study on traffic classification accuracy through FL using real-world network traffic trace and we observe serious accuracy degradation due to heterogeneous environments. We propose a novel federated analytics (FA) approach, FEAT, to improve the accuracy. Note that FL emphasizes on model training, yet our FA performs local analytic tasks that can estimate traffic data skewness and select appropriate clients for FL model training. Our analytics tasks are performed locally and in a federated manner; thus, we preserve privacy as well. Our approach has strong theoretical properties where we exploit Hoeffding inequality to infer traffic data skewness and we leverage the Thompson Sampling for client selection. We evaluate our approach through extensive experiments using real-world traffic data sets QUIC and ISCX. The extensive experiments demonstrate that FEAT can improve traffic classification accuracy in heterogeneous environments.

A general delta-based in-band network telemetry framework with extremely low bandwidth overhead

In-band network telemetry (INT) enriches network management at scale through the embedding of complete device-internal states into each packet along its forwarding path, yet such embedding of INT information also incurs significant bandwidth overhead in the data plane. We propose DeltaINT, a general INT framework that achieves extremely low bandwidth overhead and supports various packet-level and flow-level applications in network management. DeltaINTbuilds on the insight that state changes are often negligible at most time, so it embeds the complete state information into a packet only when the state change is deemed significant. We propose two variants for DeltaINTthat trade between bandwidth usage and measurement accuracy, while both variants achieve significantly lower bandwidth overhead than the original INT framework. We theoretically derive the time/space complexities and the guarantees of bandwidth mitigation for DeltaINT. We implement DeltaINTin both software and P4. Our evaluation shows that DeltaINTsignificantly mitigates the bandwidth overhead, and the deployment in a Tofino switch incurs limited hardware resource usage.

Explainable Artificial Intelligence in communication networks: A use case for failure identification in microwave networks

Artificial Intelligence (AI) has demonstrated superhuman capabilities in solving a significant number of tasks, leading to widespread industrial adoption. For in-field network-management application, AI-based solutions, however, have often risen skepticism among practitioners as their internal reasoning is not exposed and their decisions cannot be easily explained, preventing humans from trusting and even understanding them. To address this shortcoming, a new area in AI, called Explainable AI (XAI), is attracting the attention of both academic and industrial researchers. XAI is concerned with explaining and interpreting the internal reasoning and the outcome of AI-based models to achieve more trustable and practical deployment. In this work, we investigate the application of XAI for network management, focusing on the problem of automated failure-cause identification in microwave networks. We first introduce the concept of XAI, highlighting its advantages in the context of network management, and we discuss in detail the concept behind Shapley Additive Explanations (SHAP), the XAI framework considered in our analysis. Then, we propose a framework for a XAI-assisted ML-based automated failure-cause identification in microwave networks, spanning model’s development and deployment phases. For the development phase, we show how to exploit SHAP for feature selection and how to leverage SHAP to inspect misclassified instances during model’s development process, and how to describe model’s global behavior based on SHAP’s global explanations. For the deployment phase, we propose a framework based on predictions uncertainty to detect possibly wrong predictions that will be inspected through XAI.

Next Decade of Telecommunications Artificial Intelligence

It has been an exciting journey since the mobile communications and artificial intelligence were conceived 37 years and 64 years ago. While both fields evolved independently and profoundly changed communications and computing industries, the rapid convergence of 5G and deep learning is beginning to significantly transform the core communication infrastructure, network management and vertical applications. The paper first outlines the individual roadmaps of mobile communications and artificial intelligence in the early stage, with a concentration to review the era from 3G to 5G when AI and mobile communications started to converge. With regard to telecommunications artificial intelligence, the paper further introduces in detail the progress of artificial intelligence in the ecosystem of mobile communications. The paper then summarizes the classifications of AI in telecom ecosystems along with its evolution paths specified by various international telecommunications standardization bodies. Towards the next decade, the paper forecasts the prospective roadmap of telecommunications artificial intelligence. In line with 3GPP and ITU-R timeline of 5G & 6G, the paper further explores the network intelligence following 3GPP and ORAN routes respectively, experience and intention driven network management and operation, network AI signalling system, intelligent middle-office based BSS, intelligent customer experience management and policy control driven by BSS and OSS convergence, evolution from SLA to ELA, and intelligent private network for verticals. The paper is concluded with the vision that AI will reshape the future B5G or 6G landscape and we need pivot our R&D, standardizations, and ecosystem to fully take the unprecedented opportunities.

Enabling low‐latency service function chains by merging duplicate match operations

Service function chains (SFCs) built by network functions (NFs) are usually required to be low-latency, especially in some tight latency scenarios such as Distributed Denial of Service (DDoS) defense. However, different NFs in SFCs are likely to perform identical operations of matching specific packet header fields (e.g., source IP address). Such a processing redundancy inevitably increases the overall end-to-end processing latency of SFCs, which in turn affects network management applications in their decision-making process of handling short-lived network events (e.g., microbursts). To address this problem, in this paper, we propose a novel NFV framework, DMO, that aims to eliminate those identical and redundant match operations among different NFs in input SFCs while maintaining the original SFC semantics of packet processing. To achieve this goal, our design proposes a semantic-preserving mechanism that merges duplicate match operations between different NFs and also preserves original SFC semantics. Also, to avoid potential conflicts among NF rules at runtime, DMO further offers another mechanism that resolves unnecessary conflicts among different NF rules by assigning reasonable priorities to these rules. We have implemented a prototype of DMO. Our experiments indicate that DMO achieves 26.7%–67.3% latency reduction for real-world SFCs.

Multi‐hop similarity‐based‐clustering framework for IoT‐Oriented Software‐Defined wireless sensor networks

The performance of Internet of Things (IoT)-based Wireless Sensor Networks (WSNs) depends on the routing protocol and the deployment technique in modern applications. In a plethora of IoT-WSNs applications, the IoT nodes are essential equipment to prolong the network lifetime with limited resources. Data similarity-based clustering protocols exploit the temporal correlation among the neighbouring sensor nodes through the subset of data. In bendy supervision, IoT-based Software Defined WSNs provide an optimistic resolution by allowing the control logic to be separated from the sensor nodes. The benefit of this SDN-based IoT architecture, allows the unified control of the entire IoT network, making it easier to implement on-demand network management protocols and applications. To this end, in this paper, we design a Multi-hop Similarity-based Clustering framework for IoT-oriented Software-Defined wireless sensor Networks (MSCSDNs). In particular, we construct data-similar application-aware clusters in order to minimise the communication overhead. Also, we adapt inter-cluster and intra-cluster multi-hop communication using adaptive normalised least mean square and merged them with the proposed MSCSDN framework that helps prolong the network lifespan. The proposed framework is compared with the state-of-the-art approaches in terms of network lifespan, stability period, instability period, report delay, report delivery, and cluster leader nodes generations. The MSCSDN achieves optimal data accuracy concerning the collected data.

Super Spreader Identification Using Geometric-Min Filter

Super spreader identification has a lot of applications in network management and security monitoring. It is a more difficult problem than heavy hitter identification because flow spread is harder to measure than flow size due to the requirement of duplicate removal. The prior work either incurs heavy memory overhead or requires heavy computations. This paper designs a new super-spreader monitor capable of identifying all flows whose spreads are greater than a user-specified threshold with a probability that can be arbitrarily set. It introduces a generalized geometric hash function, a generalized geometric counter, and a novel geometric-min filter that blocks out the vast majority of small/medium flows from being tracked, allowing us to focus on a small number of flows in which super spreaders are identified. We provide an analytical way of properly setting the system threshold to meet probabilistically guaranteed identification of super spreaders, and implement it on both hardware (FPGA) and software platforms. We perform extensive experiments based on real Internet traffic traces from CAIDA. The results show that with proper parameter settings, the new monitor can identify more than 99% super spreaders with a low memory requirement, better than the prior art.

Design Specification for Network Fault Diagnosis System

As the network becomes more and more complex and heterogeneous, the problem of network management becomes more and more prominent. The research and application of network management is of great significance for ensuring the normal operation of network and improving the reliability and availability of network system. The main problems of network management are network configuration, failure, performance, security, planning and scaling, etc. On the basis of in-depth research and discussion on network management and fault management, this paper designs the system requirements of network fault diagnosis system.

PHeavy: Predicting Heavy Flows in the Programmable Data Plane

Since heavy flows account for a significant fraction of network traffic, being able to predict heavy flows has benefited many network management applications for mitigating link congestion, scheduling of network capacity, exposing network attacks and so on. Existing machine learning based predictors are largely implemented on the control plane of Software Defined Networking (SDN) paradigm. As a result, frequent communication between the control and data planes can cause unnecessary overhead and additional delay in decision making. In this paper, we present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pHeavy</i> , a machine learning based scheme for predicting heavy flows directly on the programmable data plane, thus eliminating network overhead and latency to SDN controller. Considering the scarce memory and limited computation capability in the programmable data plane, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pHeavy</i> includes a packet processing pipeline which deploys pre-trained decision tree models for in-network prediction. We have implemented <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pHeavy</i> in both bmv2 software switch and P4 hardware switch (i.e., Barefoot Tofino). Evaluation results demonstrate that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pHeavy</i> has achieved 85% and 98% accuracy after receiving the first 5 and 20 packets of a flow respectively, while being able to reduce the size of decision tree by 5.4x on average. More importantly, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pHeavy</i> can predict heavy flows at line rate on the P4 hardware switch.

A Machine Learning-based Approach for Network Traffic Analysis and Management

For a network to function properly and remain secure, network traffic management and analysis are essential. In this field, machine learning-based techniques have demonstrated considerable potential by offering precise and effective network traffic analysis and anomaly detection. In this research, we offer a machine learning-based methodology for network traffic monitoring and management. This method analyses network data and identifies network anomalies using a variety of machine learning methods. Using the NSL-KDD dataset and other machine learning methods, such as decision trees, SVM, neural networks, and random forests, we assess the effectiveness of our strategy. The outcomes of our tests show how successful our suggested strategy is, with high accuracy rates and low false positive rates. In numerous network management and security applications, our suggested approach beats cutting-edge machine learning-based algorithms for network traffic analysis and management. The suggested strategy offers a positive perspective for improving network administration and security through machine learning.

Performance Evaluation of Data Compression Algorithms for IoT-Based Smart Water Network Management Applications

IoT-based smart water supply network management applications generate a huge volume of data from the installed sensing devices which are required to be processed (sometimes in-network), stored and transmitted to a remote centre for decision making. When the volume of data produced by diverse IoT smart sensing devices intensify, processing and storage of these data begin to be a serious issue. The large data size acquired from these applications increases the computational complexities, occupies the scarce bandwidth of data transmission and increases the storage space. Thus, data size reduction through the use of data compression algorithms is essential in IoT-based smart water network management applications. In this paper, the performance evaluation of four different data compression algorithms used for this purpose is presented. These algorithms, which include RLE, Huffman, LZW and Shanon-Fano encoding were realised using MATLAB software and tested on six water supply system data. The performance of each of these algorithms was evaluated based on their compression ratio, compression factor, percentage space savings, as well as the compression gain. The results obtained showed that the LZW algorithm shows better performance base on the compression ratio, compression factor, space savings and the compression gain. However, its execution time is relatively slow compared to the RLE and the two other algorithms investigated. Most importantly, the LZW algorithm has a significant reduction in the data sizes of the tested files than all other algorithms

DCLab: A Reconfigurable SDN Testbed for Datacenter Networks

The SDN paradigm allows network management applications to configure and manage network devices directly. Researchers use simulators, emulation platforms, and experimental testbeds to evaluate ideas. Emulation and simulation platforms offer the advantages of more flexibility and larger scale; using a testbed produces more realistic and accurate results. Although they support SDN, the switches used in data centers pose an economic barrier for researchers. We propose DCLab that creates a low-cost reconfigurable SDN testbed using white-box switches. This letter explains the architecture of DCLab and reports experiments that demonstrate how it allows to overlay various logical topologies on a physical topology.