Traffic Matrix Research Articles

Fuzzy model of transport demand

The development of transport networks is costly and cannot solve all transport problems. In this regard, scientifically based planning of the development of the transport complex acquires an important role to improve the quality of the functioning of the transport system, such as improving the organization of traffic in various parts of the network, optimizing the system of public transport routes, creating convenient transfer hubs, etc. To solve such problems, the paper considers the development of intelligent transport systems and mathematical modeling of transport demand. The central task of developing a transport model is to determine the equilibrium state of the transport system, i.e., the problem of the distribution of flows in the network, which involves two main stages. In the first stage, one or several correspondence matrices are evaluated based on the traffic flow's initial data and behavior. The second stage consists of the distribution of traffic matrices on the graph of the transport network, i.e., in solving the problem of finding traffic flows or the problem of transport equilibrium. The problem of transport equilibrium is reduced to an optimization problem in which restrictions are imposed depending on the values of the elements of the traffic matrix. The search for new methods for solving optimization problems of weakly formalized processes is caused by the presence of processes and phenomena, which is difficult to describe by methods of classical mathematics due to their indeterminacy and unreliability, incompleteness, and fuzziness of the initial information. The mathematical model of the optimization problem is described by the objective function and constraints. Classical mathematical programming and its variations are largely a normative methodology for effective choice. Fuzzy programming highlights the natural multiplicity of inaccurately defined goals, restrictions, and their meanings.

Building a Digital Twin for network optimization using Graph Neural Networks

Network modeling is a critical component of Quality of Service (QoS) optimization. Current networks implement Service Level Agreements (SLA) by careful configuration of both routing and queue scheduling policies. However, existing modeling techniques are not able to produce accurate estimates of relevant SLA metrics, such as delay or jitter, in networks with complex QoS-aware queueing policies (e.g., strict priority, Weighted Fair Queueing, Deficit Round Robin). Recently, Graph Neural Networks (GNNs) have become a powerful tool to model networks since they are specifically designed to work with graph-structured data. In this paper, we propose a GNN-based network model able to understand the complex relationship between (i) the queueing policy (scheduling algorithm and queue sizes), (ii) the network topology, (iii) the routing configuration, and (iv) the input traffic matrix. We call our model TwinNet, a Digital Twin that can accurately estimate relevant SLA metrics for network optimization. TwinNet can generalize to its input parameters, operating successfully in topologies, routing, and queueing configurations never seen during training. We evaluate TwinNet over a wide variety of scenarios with synthetic traffic and validate it with real traffic traces. Our results show that TwinNet can provide accurate estimates of end-to-end path delays in 106 unseen real-world topologies, under different queuing configurations with a Mean Absolute Percentage Error (MAPE) of 3.8%, as well as a MAPE of 6.3% error when evaluated with a real testbed. We also showcase the potential of the proposed model for SLA-driven network optimization and what-if analysis.

Mitigating Routing Update Overhead for Traffic Engineering by Combining Destination-Based Routing With Reinforcement Learning

Traffic Engineering (TE) is a widely-adopted network operation to optimize network performance and resource utilization. Destination-based routing is supported by legacy routers and more readily deployed than flow-based routing, where the forwarding entries could be frequently updated by TE to accommodate traffic dynamics. However, as the network size grows, destination-based TE could render high time complexity when generating and updating many forwarding entries, which may limit the responsiveness of TE and degrade network performance. In this paper, we propose a novel destination-based TE solution called FlexEntry, which leverages emerging Reinforcement Learning (RL) to reduce the time complexity and routing update overhead while achieving good network performance simultaneously. For each traffic matrix, FlexEntry only updates a few forwarding entries called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">critical entries</i> for redistributing a small portion of the total traffic to improve network performance. These critical entries are intelligently selected by RL with traffic split ratios optimized by Linear Programming (LP). We find out that the combination of RL and LP is very effective. Our simulation results on six real-world network topologies show that FlexEntry reduces up to 99.3% entry updates on average and generalizes well to unseen traffic matrices with near-optimal load balancing performance.

Techniques for applying reinforcement learning to routing and wavelength assignment problems in optical fiber communication networks

We propose a novel application of reinforcement learning (RL) with invalid action masking and a novel training methodology for routing and wavelength assignment (RWA) in fixed-grid optical networks and demonstrate the generalizability of the learned policy to a realistic traffic matrix unseen during training. Through the introduction of invalid action masking and a new training method, the applicability of RL to RWA in fixed-grid networks is extended from considering connection requests between nodes to servicing demands of a given bit rate, such that lightpaths can be used to service multiple demands subject to capacity constraints. We outline the additional challenges involved for this RWA problem, for which we found that standard RL had low performance compared to that of baseline heuristics, in comparison with the connection requests RWA problem considered in the literature. Thus, we propose invalid action masking and a novel training method to improve the efficacy of the RL agent. With invalid action masking, domain knowledge is embedded in the RL model to constrain the action space of the RL agent to lightpaths that can support the current request, reducing the size of the action space and thus increasing the efficacy of the agent. In the proposed training method, the RL model is trained on a simplified version of the problem and evaluated on the target RWA problem, increasing the efficacy of the agent compared with training directly on the target problem. RL with invalid action masking and this training method outperforms standard RL and three state-of-the-art heuristics, namely, k shortest path first fit, first-fit k shortest path, and k shortest path most utilized, consistently across uniform and nonuniform traffic in terms of the number of accepted transmission requests for two real-world core topologies, NSFNET and COST–239. The RWA runtime of the proposed RL model is comparable to that of these heuristic approaches, demonstrating the potential for real-world applicability. Moreover, we show that the RL agent trained on uniform traffic is able to generalize well to a realistic nonuniform traffic distribution not seen during training, thus outperforming the heuristics for this traffic. Visualization of the learned RWA policy reveals an RWA strategy that differs significantly from those of the heuristic baselines in terms of the distribution of services across channels and the distribution across links.

Flow-by-flow traffic matrix prediction methods: Achieving accurate, adaptable, low cost results

Traffic matrix (TM) prediction methods aim to accurately and efficiently predict future network traffic demands by using previous traffic matrices. These methods are critical for network operation and management. The recent research focuses on designing complex models to extract spatiotemporal correlations from TMs and network topologies using traffic volume datasets that consist of all the origin–destination (OD) flows sampled from the network. These entire-matrix models are confined to a specific network topology and are difficult to apply to other networks. To solve such problems, we investigate a flow-by-flow prediction method in this paper which utilizes the intra-flow temporal correlations. State-of-the-art results have been achieved by our method on Abilene and GÉANT datasets. Using only 10% of the OD flows as the training samples, the flow-by-flow models can outperform the previously achieved state-of-the-art prediction performance. This small sample greatly reduces the end-to-end traffic monitoring costs. In addition, the flow-by-flow models can achieve outstanding cross-dataset evaluation performance. We suggest that leveraging a large external dataset with a flow-by-flow method can be a better choice when the measurement environment is difficult on the target network. The source code for our prediction method is publicly available at https://github.com/FreeeBird/Flow-By-Flow-Prediction.

Missing Internet Traffic Reconstruction using Compressive Sampling

Missing traffic is a commonly problem in large-scale network. Because the traffic information is needed by network engineering task for network monitoring, there are several methods that recover the missing problem. In this paper, we proposed missing internet traffic reconstruction based on compressive sampling. The main contributions of this study are as follows: (i) explore the influence of the six missing patterns on the performance of the traffic matrix reconstruction algorithm; (ii) trace the link sensitivity; and (iii) detect the time sensitivity of the network. Using Abilene data, the simulation results show that compressive sampling can perform internet traffic monitoring such as reconstruction from missing traffic, finding link sensitivity, and detecting time sensitivity.

Low-rank traffic matrix completion with marginal information

Accurate spatio-temporal traffic data is crucial to intelligent transportation systems. Missing traffic data is an important problem to solve. Low-rank matrix completion provides an effective way to find the missing data. The completion aims to obtain a low-rank matrix that can approximate the known entries as far as possible. Meanwhile, some linear constraint marginal information of the matrix can also be observed in the real application. In this paper, we utilize such marginal information to largely improve the performance of common matrix completion algorithms and propose an alternating direction method of multipliers (ADMM) and conjugate gradient descent method (CGD) based SoftImpute alternative least square (ALS) algorithm. We analyze their convergence rates and prove that the model can always converge to a first-order stationary point. We also utilize ADMM and CGD to largely accelerate the subproblem and make its complexity of each iteration at the same level as the popular SoftImpute-ALS matrix completion algorithm. Furthermore, this algorithm can be used in distributed computation, suitable for large-scale problems. In the numerical experiments, we demonstrate its outstanding matrix completion performance and high speed in several traffic matrix datasets.

SDN-Based Traffic Matrix Estimation in Data Center Networks through Large Size Flow Identification

Software defined networking (SDN) with separated control plane and data plane brings new opportunities for traffic measurement in data center networks. However, in the SDN-enabled switches, available TCAM (Ternary Content Addressable Memory) resources for traffic measurement are limited. Thus, it is necessary to utilize traffic matrix (TM) estimation to derive a hybrid network monitoring scheme through combining the partial direct measurement offered by SDN with some inference techniques. Although large size flows play an important role in improving TM estimation accuracy, directly monitoring each flow and finding out large size flows consume massive channel bandwidth resource between control plane and data plane. Therefore, in this paper, we identify large size flows from multiple historical TMs instead of monitoring each flow. First, we analyze multiple historical TMs and observe that origin-to-destination (OD) pair whose flow size is selected as large size flow at last time slot is most likely to be selected for per-flow monitoring at next time slot, so these OD pairs are identified by gradient boosting machine and are directly regarded as sampled OD pairs in order to reduce resource consumption. Then, we propose a greedy heuristic algorithm to solve SDN-enabled switch selection problem to best utilize the TCAM resources and guarantee that most of sampled OD pairs are measured in the flow table. We also present a source node prefix tree based bit merging aggregation (SPTBMA) scheme to design feasible forwarding rules to be inserted in TCAM of SDN-enabled switches and reserve more TCAM space for sampled OD pairs. Finally, the experimental results based on real traffic dataset demonstrate that our proposed scheme outperforms the existing algorithms in terms of improving TM estimation accuracy and overcoming limitation of TCAM resources.

Simultaneous Topology and Loss Tomography via a Modified Theme Dictionary Model

As a technique to investigate behaviours of a computer network with low operational cost, network tomography has received considerable attentions in recent years. Most studies in this area assume that the topology of the network of interest is known, and try to propose computationally and/or statistically efficient methods to estimate link-level properties such as loss rate, delay distribution, bandwidth etc., or global traffic properties such as point-to-point traffic matrix. Little progresses have been made for scenarios when topology of the target network is unknown, although it is often the case in many practical applications. The few published works for topology tomography resolved the problem primarily by clustering analysis, which works for tree-like networks only and often suffers from unstable performance for large networks of complicated structure. In this article, we study the classic problem of network tomography from a new perspective. By connecting the problem of topology tomography to the classic machine learning problem of &#x201C;market basket analysis,&#x201D; we find that simultaneous topology and loss tomography can be achieved by discovering association patterns of loss records collected at receivers, which can be efficiently resolved with light modifications of a recently developed statistical method known as the &#x201C;theme dictionary model&#x201D;. Both theoretical analysis and simulation studies demonstrate that the proposed approach enjoys improved effectiveness for networks of tree as well as general topology with slightly higher computational costs.

Should Traffic Matrix Prediction Be Performed Entirely?

Should Traffic Matrix Prediction Be Performed Entirely?

Should Traffic Matrix Prediction Be Performed Entirely?

Should Traffic Matrix Prediction Be Performed Entirely?

Multipath Wireless Network Coding: A Scheduling Constraint in Game Perspective

This paper considers wireless networks in which various paths are obtainable involving each source and destination. It is allowing each source to tear traffic among all of its existing paths, and it may conquer the lowest achievable number of transmissions per unit time to sustain a prearranged traffic matrix. Traffic bound in contradictory instructions in excess of two wireless hops can utilize the “reverse carpooling” advantage of network coding in order to decrease the number of transmissions used. These call such coded hops “hyper-links.” With the overturn carpooling procedure, longer paths might be cheaper than shorter ones. However, convenient is an irregular situation among sources. The network coding advantage is realized only if there is traffic in both directions of a shared path. This project regard as the problem of routing amid network coding by egotistic agents (the sources) as a potential game and develop a method of state-space extension in which extra agents (the hyper-links) decouple sources’ choices from each other by declaring a hyper-link capacity, allowing sources to split their traffic selfishly in a distributed fashion, and then altering the hyper-link capacity based on user actions. Furthermore, each hyper-link has a scheduling constraint in stipulations of the maximum number of transmissions authorized per unit time. Finally these project show that our two-level control scheme is established and verify our investigative insights by simulation.

Green distributed algorithm for energy saving in IP wired networks using sleep scheduling

Energy saving has become a critical issue and a great challenge in the past few decades, and a great effort as well is being made to reduce consumed energy. The Internet forms a major source for energy consumption. Therefore, in this work we propose an algorithm for energy saving in distributed backbone networks, the reduced energy consumption (RedCon) algorithm. In this paper, we introduce a new version for saving energy on the Internet by switching off underutilized links and switching on idle links when the network is overloaded in a distributed manner over the network nodes based on LSA messages and without any knowledge of the traffic matrix. Our algorithm is more accurate and outperforms other algorithms with its time checks and advanced learning algorithm.

Birkhoff’s Decomposition Revisited: Sparse Scheduling for High-Speed Circuit Switches

Data centers are increasingly using high-speed circuit switches to cope with the growing demand and reduce operational costs. One of the fundamental tasks of circuit switches is to compute a sparse collection of switching configurations to support a traffic demand matrix. Such a problem has been addressed in the literature with variations of the approach proposed by Birkhoff in 1946 to decompose a doubly stochastic matrix exactly. However, the existing methods are heuristic and do not have theoretical guarantees on how well a collection of switching configurations (i.e., permutations) can approximate a traffic matrix (i.e., a scaled doubly stochastic matrix). In this paper, we revisit Birkhoff's approach and make three contributions. First, we establish the first theoretical bound on the sparsity of Birkhoff's algorithm (i.e., the number of switching configurations necessary to approximate a traffic matrix). In particular, we show that by using a subset of the admissible permutation matrices, Birkhoff's algorithm obtains an $\epsilon$-approximate decomposition with at most $O( \log(1 / \epsilon))$ permutations. Second, we propose a new algorithm, Birkhoff+, which combines the wealth of Frank-Wolfe with Birkhoff's approach to obtain sparse decompositions in a fast manner. And third, we evaluate the performance of the proposed algorithm numerically and study how this affects the performance of a circuit switch. Our results show that Birkhoff+ is superior to previous algorithms in terms of throughput, running time, and number of switching configurations.

Structured nonnegative matrix factorization for traffic flow estimation of large cloud networks

Network traffic matrix estimation is an ill-posed linear inverse problem: it requires to estimate the unobservable origin destination traffic flows, X, given the observable link traffic flows, Y, and a binary routing matrix, A, which are such that Y=AX. This is a challenging but vital problem as accurate estimation of OD flows is required for several network management tasks. In this paper, we propose a novel model for the network traffic matrix estimation problem which maps high-dimension OD flows to low-dimension latent flows with the following three constraints: (1) nonnegativity constraint on the estimated OD flows, (2) autoregression constraint that enables the proposed model to effectively capture temporal patterns of the OD flows, and (3) orthogonality constraint that ensures the mapping between low-dimensional latent flows and the corresponding link flows to be distance preserving. The parameters of the proposed model are estimated with a training algorithm based on Nesterov accelerated gradient and generally shows fast convergence. We validate the proposed traffic flow estimation model on two real backbone IP network datasets, namely Internet2 and GÉANT. Empirical results show that the proposed model outperforms the state-of-the-art models not only in terms of tracking the individual OD flows but also in terms of standard performance metrics. The proposed model is also found to be highly scalable compared to the existing state-of-the-art approaches.

Internet traffic matrix prediction with convolutional LSTM neural network

With the rapid growing trend of Internet, prediction‐based network operation optimization and management has drawn the attention from both the academia and the industry. For predicting Internet traffic, deep learning models have been proven more effective than linear and statistical models. However, some of the previous studies model the Internet traffic prediction as a multivariate time series prediction problem simply, without using the Internet traffic matrix structure. In this letter, the Internet traffic matrix prediction problem is firstly modeled as a video prediction task. Then a ConvLSTM‐based Seq2Seq model named ConvLSTM‐TM is proposed for predicting the traffic matrix in the next time slot. Based on three real‐world traffic matrix datasets, ConvLSTM‐TM outperforms five deep learning baselines with a lower prediction error. For replication, the dataset, code, and results are publicly available in a Github repository.

Reducing power consumption in LEO satellite network

Current low earth orbit (LEO) satellite network display poor power efficiency, running network devices at full capacity all the time regardless of the traffic matrix and the distribution of the population over the Globe. Most of the research on energy efficiency of LEO satellites has focused on component level or link level. Therefore, this kind of research is not holistic to try to look at the satellite system as a single node. To enhance the energy efficiency. The solution should exploits multipath routing and load balancing. LEO network is overprovisioned, and hence selectively shutting down some satellite nodes and links during off-peaks hours seems like a good way to reduce energy consumption. In this paper, we exploit the fact that due to geographical and climatic conditions, some satellite links are expected to be loaded with data while others remain unused. Our approach is to power down satellite nodes and links during period of low traffic, while guaranteeing the connectivity and QoS. Finding the optimal solution is NP-problem and therefore, we explore in this work two heuristic algorithms. We evaluate our heuristics on a realistic LEO topology and real traffic matrices. Simulation results show that the power saving can be significant.

ORP: An Online Rule Placement Scheme to Optimize the Traffic Overhead for Data Center Networks

In cloud data centers, the requirement for fine-grained policy leads to a large number of server rules. However, the servers cannot efficiently handle all of these rules due to limitations of the central processing unit (CPU) and memory resources. Thus, a rule partition and placement algorithms have been proposed to divide the original ruleset into several rule subsets and migrate some rule subsets to other devices. Correspondingly, the traffic, which is meant to be blocked by the original server's migratory rules, will also be redirected to the new device. The rule partition and placement algorithms should minimize this extra traffic overhead. The state-of-the-art algorithm, vCRIB, is offline, as its solution is calculated based on a static traffic matrix. In reality, when facing dynamic traffic, the vCRIB solution may actually be suboptimal in terms of traffic overhead. To reduce the traffic overhead, we propose the online rule replacement (ORP) algorithm. Specifically, ORP utilizes the difference between the old and new traffic matrices to discover the rule subsets matching the notably changed traffic overhead or blocked traffic. These rule subsets are then preferentially adjusted to their proper placement. In this manner, ORP can promptly make rule subset adjustments according to the newly updated traffic matrix. Additionally, ORP effectively avoids the invalid migrations existing in vCRIB. The simulation results confirm that ORP is capable of reducing traffic overhead by 26.67%–56.9% compared to vCRIB.

SOSW: scalable and optimal nearsighted location selection for fog node deployment and routing in SDN-based wireless networks for IoT systems

In a fog computing (FC) architecture, cloud services migrate towards the network edge and operate via edge devices such as access points (AP), routers, and switches. These devices become part of a virtualization infrastructure and are referred to as “fog nodes.” Recently, software-defined networking (SDN) has been used in FC to improve its control and manageability. The current SDN-based FC literature has overlooked two issues: (a) fog nodes’ deployment at optimal locations and (b) SDN best path computation for data flows based on constraints (i.e., end-to-end delay and link utilization). To solve these optimization problems, this paper suggests a novel approach, called scalable and optimal near-sighted location selection for fog node deployment and routing in SDN-based wireless networks for IoT systems (SOSW). First, the SOSW model uses singular-value decomposition (SVD) and QR factorization with column pivoting linear algebra methods on the traffic matrix of the network to compute the optimal locations for fog nodes, and second, it introduces a new heuristic-based traffic engineering algorithm, called the constraint-based shortest path algorithm (CSPA), which uses ant colony optimization (ACO) to optimize the path computation process for task offloading. The results show that our proposed approach significantly reduces average latency and energy consumption in comparison with existing approaches.

Gateway Placement Optimization in LEO Satellite Networks Based on Traffic Estimation

Using satellite constellations to provide global Internet access services has recently drawn increasing attention. A low-Earth orbit (LEO) satellite network with multiple satellites provides global coverage, low latency, and operates independently, by which it effectively complements terrestrial IP networks. Satellite gateways are located on the ground and can serve as data exchange points between satellite networks and the Internet. As the placement scheme can affect network performance, finding appropriate sites for gateways constitutes a fundamental problem. This paper proposes a gateway placement optimization (GPO) method for LEO satellite networks in order to solve this problem by modeling it as a combination optimization problem. We aim to identify the best gateway locations that can balance traffic loads while using as few gateways as possible. The constraints to be satisfied concern the physical links between gateways and satellites: specifically, link interference, satellite bandwidth, and number of satellite antennas. We use a gravity model to estimate the traffic matrix from/to gateways and satellites, then we adopt and modify the discrete particle swarm optimization (PSO) algorithm to solve this problem. Finally, we apply the GPO method to numerical tests on real satellite constellations. The results indicate that our method performs well and effectively.