Network Traffic Estimation Research Articles

Traffic Count Estimation at Basis Links Without Path Flow and Historic Data

Traffic counts (or link counts) are defined as cumulative traffic in the lanes between two consecutive intersections on a road network. Established methods of link count estimation assume the availability of count data at a set of basis links: a minimum subset of all the links of a network that still allow complete network traffic count estimation. If traffic count data are missing even at some basis links, current research must introduce additional assumptions, on path flow or historical data, to compensate. In this research, we present an approach to estimate the missing basis link count without the need for historical data or path-flow information, thereby overcoming the limitations of state-of-the-art estimation approaches. We develop a stochastic method using a canonical correlation analysis-based constrained minimization problem for estimation purposes. The proposed method has been validated with real-world link count data collected in Melbourne, Australia, between <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2016$</tex-math> </inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2019$</tex-math> </inline-formula> . The validation results indicate that we can achieve an accuracy of up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$90\%$</tex-math> </inline-formula> in the real world, despite the unknown traffic patterns of the estimation period. Depending on the time of day, the modelling strategy selected, and the consistency of input data available at the road intersection, the estimation accuracy varies. The proposed methodology is useful when there is a general shortage of data since there is inadequate infrastructure for data collection in many major cities around the world.

Traffic based power consumption and node deployment in green LTE-A cellular networks

The energy utilization of green cellular networking or cellular network infrastructure has become a popular research domain due to the economic concern of network operators and global climate change. Also, due to the growth of wireless networks, the authors are attracted to finding novel relay station (RS) deployment techniques to support high user density and high data rate services by reducing energy consumption and operating costs of different network elements. The location of RSs greatly influences how far a cell can be covered. Therefore, the deployment RS plays a major role in widening the cell's coverage radius. To solve these challenges and to realize greener network designs by maximizing power consumption with QoS assurance, a novel traffic-based power consumption minimization and node deployment for the LTE green cellular networks is proposed. Also, key performance indicators (KPIs) are required to monitor and improve network performance. KPIs control the accomplished resource utilization and the quality of services. The research work comprises RS deployment, traffic estimation, and power allocation. Power losses, communication delays, increased implementation costs, and decreased throughput are the effects of an inappropriate RS deployment. Therefore, this research introduces the transmission area-based relay stations deployment scheme (TARSD) with multiobjective functions maximization of coverage area, minimization of the overlapping area and the minimization of power consumption cost. Traffic estimation helps to identify future capacity requirements and enables improved planning and decision-making. The deployment scheme of a cellular network relies on network traffic estimation. Nash traffic entropy learning algorithm (NTEL) is used to estimate traffic, and based on the estimated eNB switching ON/OFF is performed to minimize power consumption. Inter cell interference (ICI) can impact the performance of the LTE network. Several power allocation schemes can be used to improve system performance to achieve the optimal trade-off between interference and signal-to-noise ratio (SINR). Because of its good performance and efficacy in improving the mutual information between a channel's input and output, the enhanced low complexity water-filling (ELCWF) has been widely employed in power allocation with its increment and decrement properties. This research studies the statistical behaviour of a Received Signal Strength Indicator (RSSI), Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ) and Signal to Interference noise Ratio (SINR). Also, statistical analysis of cost versus coverage performance is examined. Under the 7th simulation run, the proposed method yields the best coverage of 88.81 and 98.74%, respectively. In the experimental scenario, for the proposed model, the deployment cost for BS and RS are 27 and 13 units, the average throughput per user is 9.74 Mbps, and the coverage ratio is 97% under the BS candidate location of 20, respectively. In the power allocation stage, if the number of users is 50, the proposed model yields a power consumption of 0.2 W, throughput of 23.4 Mbps, delay of 15 ms and SINR of 20.12 dB, respectively.

Congestion Prediction in Internet of Things Network using Temporal Convolutional Network A Centralized Approach

The unprecedented ballooning of network traffic flow, specifically, Internet of Things (IoT) network traffic, has big stressed of congestion on todays Internet. Non-recurring network traffic flow may be caused by temporary disruptions, such as packet drop, poor quality of services, delay, etc. Hence, the network traffic flow estimation is important in IoT networks to predict congestion. As the data in IoT networks is collected from a large number of diversified devices which have unlike format of data and also manifest complex correlations, so the generated data is heterogeneous and nonlinear in nature. Conventional machine learning approaches unable to deal with nonlinear datasets and suffer from misclassification of real network traffic due to overfitting. Therefore, it also becomes really hard for conventional machine learning tools like shallow neural networks to predict the congestion accurately. Accuracy of congestion prediction algorithms play an important role to control the congestion by regulating the send rate of the source. Various deeplearning methods (LSTM, CNN, GRU, etc.) are considered in designing network traffic flow predictors, which have shown promising results. In this work, we propose a novel congestion predictor for IoT, that uses Temporal Convolutional Network (TCN). Furthermore, we use Taguchi method to optimize the TCN model that reduces the number of runs of the experiments. We compare TCN with other four deep learning-based models concerning Mean Absolute Error (MAE) and Mean Relative Error (MRE). The experimental results show that TCN based deep learning framework achieves improved performance with 95.52% accuracy in predicting network congestion. Further, we design the Home IoT network testbed to capture the real network traffic flows as no standard dataset is available.

Tracking Normalized Network Traffic Entropy to Detect DDoS Attacks in P4

Distributed Denial-of-Service (DDoS) attacks represent a persistent threat to modern telecommunications networks: detecting and counteracting them is still a crucial unresolved challenge for network operators. DDoS attack detection is usually carried out in one or more central nodes that collect significant amounts of monitoring data from networking devices, potentially creating issues related to network overload or delay in detection. The dawn of programmable data planes in Software-Defined Networks can help mitigate this issue, opening the door to the detection of DDoS attacks directly in the data plane of the switches. However, the most widely-adopted data plane programming language, namely P4, lacks supporting many arithmetic operations, therefore, some of the advanced network monitoring functionalities needed for DDoS detection cannot be straightforwardly implemented in P4. This work overcomes such a limitation and presents two novel strategies for flow cardinality and for normalized network traffic entropy estimation that only use P4-supported operations and guarantee a low relative error. Additionally, based on these contributions, we propose a DDoS detection strategy relying on variations of the normalized network traffic entropy. Results show that it has comparable or higher detection accuracy than state-of-the-art solutions, yet being simpler and entirely executed in the data plane.

Structured nonnegative matrix factorization for traffic flow estimation of large cloud networks

Network traffic matrix estimation is an ill-posed linear inverse problem: it requires to estimate the unobservable origin destination traffic flows, X, given the observable link traffic flows, Y, and a binary routing matrix, A, which are such that Y=AX. This is a challenging but vital problem as accurate estimation of OD flows is required for several network management tasks. In this paper, we propose a novel model for the network traffic matrix estimation problem which maps high-dimension OD flows to low-dimension latent flows with the following three constraints: (1) nonnegativity constraint on the estimated OD flows, (2) autoregression constraint that enables the proposed model to effectively capture temporal patterns of the OD flows, and (3) orthogonality constraint that ensures the mapping between low-dimensional latent flows and the corresponding link flows to be distance preserving. The parameters of the proposed model are estimated with a training algorithm based on Nesterov accelerated gradient and generally shows fast convergence. We validate the proposed traffic flow estimation model on two real backbone IP network datasets, namely Internet2 and GÉANT. Empirical results show that the proposed model outperforms the state-of-the-art models not only in terms of tracking the individual OD flows but also in terms of standard performance metrics. The proposed model is also found to be highly scalable compared to the existing state-of-the-art approaches.

Urban network traffic state estimation using a data-based approach

In this paper we propose an estimator of vehicle density in every road section of a large urban traffic network. We assume a limited number of flow and turning ratio sensors can be installed, and that aggregate floating car data (FCD) is available, such that the space-mean speed of each road can be estimated. We propose a method to locate turning ratio sensors, which takes as input previous low-quality estimates of the turn rates, and then assigns to each intersection a weight according to the effect on the total density reconstruction error caused by perturbations between a priori and actual turning ratio values. We evaluate the models and estimator using data from the urban traffic network of Grenoble in France.

Anomaly-Aware Network Traffic Estimation via Outlier-Robust Tensor Completion

Accurately estimating network traffic from the partial measurements plays a crucial role in network management. However, the potential anomaly existing in real networks usually makes this goal difficult to achieve. Existing network traffic estimation methods generally impute network traffic independent of anomaly detection, which incurs significant performance degradation with network anomaly. To address this issue in the realistic network scenario, we propose a novel anomaly-aware network traffic estimation method to recover network traffic data concurrently with network anomaly detection. Specifically, by exploiting the inherent spatio-temporal characteristics, we first formulate the network traffic estimation as a low-rank tensor completion problem. Then, an outlier-robust tensor completion (OrTC) model is constructed by introducing both L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm regularization and L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</sub> -norm regularization, which can not only well fit the intrinsic low-rank property of real traffic data, but also is robust against both the dense noise and the sparse anomaly. Furthermore, an effective optimization algorithm OrTC-AM is designed to solve the non-convex and non-smooth OrTC model based on the popular alternating minimization method. Finally, the extensive experiments performed on the public dataset demonstrate that our proposed OrTC-AM method outperforms the previously widely used network traffic estimation methods.

Approximation of regression-based fault minimization for network traffic

This research associates three distinct approaches for computer network traffic prediction. They are the traditional stochastic gradient descent (SGD) using a few random samplings instead of the complete dataset for each iterative calculation, the gradient descent algorithm (GDA) which is a well-known optimization approach in Deep Learning, and the proposed method. The network traffic is computed from the traffic load (data and multimedia) of the computer network nodes via the Internet. It is apparent that the SGD is a modest iteration but can conclude suboptimal solutions. The GDA is a complicated one, can function more accurate than the SGD but difficult to manipulate parameters, such as the learning rate, the dataset granularity, and the loss function. Network traffic estimation helps improve performance and lower costs for various applications, such as an adaptive rate control, load balancing, the quality of service (QoS), fair bandwidth allocation, and anomaly detection. The proposed method confirms optimal values out of parameters using simulation to compute the minimum figure of specified loss function in each iteration.

Sparse Bayesian Learning Assisted Approaches for Road Network Traffic State Estimation

Real-time (online) road network traffic state estimation plays a vital role in enhancing the services offered by Intelligent Transportation Systems (ITS). Spatio-temporal data vacancies of contemporary acquisition systems considerably limit the reliability of online traffic state estimation. Online estimation insists smaller temporal frame widths inhibiting the accuracies of low rank matrix reconstruction approaches, while matrix imputation methods do not capture the non-trivial traffic data relationships. Alternatively, this paper investigates traffic state estimation by constructing sparse representations via Sparse Bayesian Learning (SBL) and Block SBL (BSBL) approaches to accommodate under-sampled data, independent of the acquisition type. Appropriate kernel matrices are determined by leveraging historical spatio-temporal correlations among the road network traffic data. Subsequently, unavailable traffic states are estimated from predictive distributions. The estimates are further pruned by kalman filter that corroborates online processing. With the SBL approach, experiments on PeMS traffic data demonstrate less than 6% Normalized Mean Absolute Error (NMAE) for a Signal Integrity (SI) of 0.5. Compared to the state-of-the-art approaches, this value is significantly better. BSBL approach gives similar error performance as SBL despite a SI of 0.26, at the cost of increased computational time. The NMAE from kalman filtered SBL (SBL+K) approach is less than 3.5% in contrast to 4.5% from kalman filtered BSBL (KBSBL) approach, thus demonstrating SBL+K approach as a good compromise between NMAE and computational time, facilitating more accurate online traffic state estimation.

Traffic State Estimation in Heterogeneous Networks with Stochastic Demand and Supply: Mixed Lagrangian–Eulerian Approach

A network fundamental diagram (NFD) represents the relationship between network-wide average flow and average density. Network traffic state estimation to observe NFD when congestion is heterogeneously distributed, as a result of a time-varying and asymmetric demand matrix, is a challenging problem. Recent studies have formulated the NFD estimation problem using both fixed measurements and probe trajectories. They are often based on a given ground-truth NFD for a single day demand. Stochastic variations in network demand and supply may significantly affect the approximation of an NFD. This study proposes a modified framework to estimate network traffic states to observe NFD while capturing the stochasticity in transportation networks. A mixed integer problem with non-linear constraints is formulated to address stochasticity in the NFD estimation problem. To solve this Nondeterministic Polynomial-hard (NP-hard) problem, a solution algorithm based on the simulated annealing method is applied. The problem is formulated and the solution algorithm is implemented to find an optimal configuration of loop detectors and probe vehicles to estimate the NFD of the Chicago downtown network and capture its day-to-day variations, considering a given available budget. Ground-truth NFDs and estimated NFDs based on a subset of loop detectors and probe vehicles are calculated using a simulation-based dynamic traffic assignment model, which is the best surrogate available to replicate real-world conditions. The main contribution of this study is to capture stochasticity in the demand and supply sides to find a more robust subset of links and trajectories to be acquired for the NFD estimation.

Anomaly-Tolerant Network Traffic Estimation via Noise-Immune Temporal Matrix Completion Model

Accurately estimating origin-destination (OD) network traffic is crucial for network management and capacity planning. However, the potential network anomaly and complex noise make this goal difficult to achieve. Existing network traffic estimation methods usually impute network traffic independent of anomaly detection, which ignores the potential relationship between the two tasks to help each other in achieving better performance. Moreover, these approaches can only be suitable for simple Gaussian or outlier noise assumptions, which cannot be applied to more complex noise distributions in practical applications. To address these issues, we propose a novel anomaly-tolerant network traffic estimation approach for simultaneously estimating network traffic and detecting network anomaly. Specifically, by utilizing the inherent low-rank property and temporal characteristic of traffic matrix, we formulate the network traffic estimation problem as a noise-immune temporal matrix completion (NiTMC) model, where the complex noise is fitted by mixture of Gaussian (MoG), and the network anomaly is smoothed by the $L_{2,1}$ -norm regularization. In addition, we also design a convergence-guaranteed optimization algorithm based on the expectation maximization (EM) and block coordinate update (BCU) methods to solve the proposed model. Furthermore, to deal with large-scale network problems, we develop a scalable and memory-efficient algorithm by employing stochastic proximal gradient descent (SPGD) method. Finally, the extensive experiments performed on real datasets demonstrate that our proposed NiTMC model outperforms the previously widely used network traffic estimation methods.

A Novel Efficient Traffic Estimation Strategy Using Hello Messaging Through Local Link Connectivity in MANETs

Network traffic estimation under random mobility of network nodes is one of the key challenges for effective communication in mobile ad hoc networks (MANETs). Under random movement of network nodes and uniform distribution of path durations among neighboring nodes, there must be a unified model to determine an adequate network traffic estimation strategy in MANETs. In this paper, a local link connectivity model that guarantees effective communication among the neighboring nodes inside a cluster is presented. Furthermore, a novel efficient traffic estimation (ETE) strategy is presented using a Markov chain process and Hello messaging through local link connectivity among neighboring nodes under uniform speed and random trajectories of mobile nodes. It has been observed that the ETE strategy is affected by the transmission rate of Hello messages, link probability at current time step of the Markov process, expected number of nodes inside a cluster, received signal strength, and probability of successful reception of the expected Hello messages at next time step of the Markov process. An expected increase in the critical transmission range of a network node as a result of possible overlapping with its neighboring nodes inside the cluster due to random mobility has also been investigated. Analytical and simulation results demonstrate efficacy of the proposed ETE strategy in terms of successful reachability of transmitted Hello messages, mean connection time, mean CH degree, queuing delay, and load balancing. In addition, the ETE strategy achieves better local link connectivity among neighboring nodes, lower Hello messages transmission latency, and better throughput in terms of successful reception of Hello messages by a neighboring node.

Spatio-Temporal Network Traffic Estimation and Anomaly Detection Based on Convolutional Neural Network in Vehicular Ad-Hoc Networks

Over the last decade, vehicular ad-hoc networks (VANETs) have received a greater attention in academia and industry due to their influence in intelligent transportation systems. Providing reliability and security to the VANET is essential in order to guarantee the efficiency of its applications. Anomaly detection has become a challenging problem due to the unique environment of VANETs with quick movement and short-lived link. In this paper, a method using the spatio-temporal feature of network traffic is proposed to implement network traffic estimation at first, and on this basis, an anomaly detection algorithm is put forward. The convolutional neural network is employed to extract the spatio-temporal features of the traffic matrix. In terms of the extracted features, network traffic is estimated by using a fully connected architecture as the output layer. Then, a threshold-based separation method is used to implement anomaly detection. The preliminary experiments comparing the proposed method with other machine learning-based methods show the effectiveness of the proposed anomaly detection method.

Traffic matrix prediction and estimation based on deep learning in large-scale IP backbone networks

Network traffic analysis has been one of the most crucial techniques for preserving a large-scale IP backbone network. Despite its importance, large-scale network traffic monitoring techniques suffer from some technical and mercantile issues to obtain precise network traffic data. Though the network traffic estimation method has been the most prevalent technique for acquiring network traffic, it still has a great number of problems that need solving. With the development of the scale of our networks, the level of the ill-posed property of the network traffic estimation problem is more deteriorated. Besides, the statistical features of network traffic have changed greatly in terms of current network architectures and applications. Motivated by that, in this paper, we propose a network traffic prediction and estimation method respectively. We first use a deep learning architecture to explore the dynamic properties of network traffic, and then propose a novel network traffic prediction approach based on a deep belief network. We further propose a network traffic estimation method utilizing the deep belief network via link counts and routing information. We validate the effectiveness of our methodologies by real data sets from the Abilene and GÉANT backbone networks.

Modeling network traffic for traffic matrix estimation and anomaly detection based on Bayesian network in cloud computing networks

With the rapid development of a cloud computing network, the network security has been a terrible problem when it provides much more services and applications. Network traffic modeling and analysis is significantly crucial to detect some lawless activities such as DDoS, virus and worms, and so on. Meanwhile, it is a common approach for acquiring a traffic matrix, which can be used by network operators to carry out network management and planning. Although a great number of methods have been proposed to model and analyze the network traffic, it is still a remarkable challenge since the network traffic characterization has been tremendously changed, in particular, for a cloud computing network. Motivated by that, we analyze and model the statistical features of network traffic based on the Bayesian network in this paper. Furthermore, we propose an accurate network traffic estimation approach and an efficient anomaly detection approach, respectively. In detail, we design a Bayesian network structure to model the causal relationships between network traffic entries. Based on this Bayesian network model, we obtain a joint probability distribution of network traffic by the maximum a posteriori approach. Then, we estimate the network traffic in terms of a regularized optimization model. Meanwhile, we also perform anomaly detection based on the proposed Bayesian network structure. We finally discuss the effectiveness of the proposed method for traffic matrix estimation and anomaly detection by applying it to the Abilene and GEANT networks.

A compressive sensing‐based approach to end‐to‐end network traffic reconstruction utilising partial measured origin‐destination flows

AbstractTraffic matrices describe the volume of traffic flowing from origin nodes to destination nodes. As one of the most crucial input parameters for network activities, it is employed to perform a great number of network management tasks. However, traffic matrix estimation is unavailable and has a highly ill‐posed nature, which means that it is significantly difficult to ensure the accuracy for estimating traffic matrices. Thus, it is a challenge to obtain the precise end‐to‐end network traffic. In this paper, we investigate the problem of network traffic estimation and propose a novel compressive sensing‐based approach using partial measured origin‐destination (OD) flows. As a purely data‐driven method, we first propose a strategy to select a small subset of the measured OD flows. We then put forward an optimal greedy adaptive dictionary learning algorithm in order to make the traffic matrix sparse. Furthermore, we model the traffic matrix estimation problem by a fundamental inference problem, which is ill‐posed and obeys the constraints of compressive sensing. In other words, the estimation model is a convex optimisation problem. In the process, we validate the performance of our method by real traffic data. Simulation results indicate that our method can trace the shape of each OD flow much more precisely. Copyright © 2014 John Wiley &amp; Sons, Ltd.

빅데이터 분석 방법을 적용한 스마트 TV의 발전 방안에 관한 제언

스마트 TV는 지상파방송 시청은 물론 인터넷에 연결되어 VOD, 게임, 영상통화, 애플리케이션 활용 등 컴퓨터 활용이 가능한 TV이다. 스마트 TV 사업을 수행하기 위해서는 콘텐츠, 플랫폼, 네트워크, 단말기 등이 모두 포함 되어야 한다. 만약 이중 한 가지라도 갖추어 지지 않으면 다른 사업자와 협력해야 한다. 따라서 스마트 TV 사업에 있어서 각각사업 주체들 간의 협력이 반드시 필요하다. 본 논문에서는 국내외 스마트 TV 시장, 정책, 활성화 전략 등에 대하여 살펴본다. 그리고 스마트 TV 활성화 방안인 1)클라우드 기반의 하드웨어 인프라 구축, 2)트래픽 증가를 수용하는 네트워크 고도화, 3)각 사업자들 간의 기술개발 공조, 4)다양한 스마트 TV 콘텐츠 제공, 5)N-스크린을 위한 UX/UI의 활용 등에 있어서 이해 당사자들의 공조, 증가할 네트웍크의 트래픽 추정, 소비자에게 실시간 스마트한 콘텐츠의 맞춤식 제공 등을 위하여 빅데이터 분석 방법론의 적용을 제안한다. A smart TV is able to show terrestrial broadcasting and also can be used as a computer -VOD, games, image communications, application utilities and so on. In order to carry out Smart TV business, it has to contains contents, platforms, network terminal unit. If ill-equipped with any of these aboves, it must cooperate with other licensee. Therefore, Smart TV business is necessary to cooperate with each business agent. In this paper, we will look into domestic/foreign country Smart TV market, policy, vitalization strategy, and suggest the application of big data analysis methodology for Smart TV vitalization method - 1) hardware infrastructure building based on cloud computing 2) Network upgradability acceptable traffic increase 3) Technical development cooperation between each licensee 4) Variable Smart TV contents supply 5) Cooperation with party interested individuals in using UX/UI for N-Screen, network traffic estimation may increase, customized supply smart contents for consumer in real time.

Probability-model based network traffic matrix estimation

Traffic matrix is of great help in many network applications. However, it is very difficult to estimate the traffic matrix for a large-scale network. This is because the estimation problem from limited link measurements is highly underconstrained. We propose a simple probability model for a large-scale practical network. The probability model is then generalized to a general model by including random traffic data. Traffic matrix estimation is then conducted under these two models by two minimization methods. It is shown that the Normalized Root Mean Square Errors of these estimates under our model assumption are very small. For a large-scale network, the traffic matrix estimation methods also perform well. The comparison of two minimization methods shown in the simulation results complies with the analysis.

Integrating Behavioral Models in Network Operations

The main goal of the study was to integrate demand models into weather-responsive network traffic estimation and prediction system methodologies. The study examined the behavioral responses of travelers along several dimensions in response to weather-related transportation management strategies in conjunction with active travel demand management strategies before and during severe weather events. Specific management interventions included pretrip, information-based mode, and departure time choice adjustments, as well as policy-based rescheduling of school hours. The paper presents a case study of the Chicago, Illinois, area network under snow conditions to assess the effect of a combination of demand management strategies to maintain the same level of network performance as under clear weather conditions. A combination of earlier dissemination of information and school-opening policy resulted in a similar level of network performance maintained under a median snow day as compared with a clear weather day. The paper presents integrated supply and demand models for simulation and an assessment of demand management strategies in conjunction with weather-related congestion.

An End-to-End Traffic Estimation Algorithm via Multifractal Wavelet Model and Principal Component Analysis in Smart Grid

In this paper, we study the problem of end-to-end network traffic estimation in the electric power communication network. Based on principal component analysis and multifractal wavelet model, we propose an effective and precise approach to estimate each origin-destination pair. Numeral results state that our method can capture the changes of end-to-end network traffic accurately in electric power communication network.